Projection apparatus for indicating a recommended position to observe a movable body, portable device, and recording medium

A projection apparatus for use with a movable body is provided. The projection apparatus includes a parking controller configured to cause the movable body to automatically move to a parking space, and a projector configured to project light onto a road surface in vicinity of the movable body. When the parking controller causes the movable body to automatically park in the parking space, the projector projects the light onto the road surface located within a predetermined distance from the movable body.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority to Japanese Patent Application No. 2018-216843, filed on Nov. 19, 2018, and Japanese Patent Application No. 2019-131368, filed on Jul. 16, 2019, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The disclosures herein relate to a projection apparatus for use with a movable body, a portable device, and a recording medium.

2. Description of the Related Art

A technology that projects a laser beam indicating the moving direction of a vehicle onto a road surface is known (see Patent Document 1, for example). Patent Document 1 describes a vehicle safety support device that projects a laser beam indicating a predicted driving trajectory of a vehicle onto a road surface.

According to the above technology, not only a driver of the movable body, but also people in the vicinity of the movable body are able to determine the moving direction of the movable body. For example, the driver is able to predict that the movable body may contact an obstacle at the time of parking. If there is a person in the vicinity of the movable body, the person is able to determine the moving direction of the movable body and move away from the movable body.

In recent years, technologies related to autonomous driving have been developed, in which a driver is not required to drive a movable body, and the movable body automatically drives itself while detecting surrounding conditions. In such autonomous driving, even if a driver (hereinafter referred to as an owner of a movable body as the driver is not necessarily required to drive the movable body) is not in the movable body, the movable body may move autonomously. Therefore, there are often cases where the owner of the movable body observes the autonomously moving movable body from the outside of the movable body.

In such cases, the owner desires to determine a suitable position to observe the autonomously moving movable body while predicting the moving direction of the movable body. In response to such a demand, as in the related-art technology, it is considered to project a predicted driving trajectory of the movable body onto the road surface. However, it would be difficult for the owner to determine a suitable position to observe the movable body based only on the driving trajectory.

For example, there may be situations where a movable body is parked automatically or is automatically driven to the entrance of the owner's house after leaving a garage. Movable bodies capable of moving autonomously are equipped with various sensors, but may sometimes fail to detect small obstacles. In such a case, if the owner of a movable body observes the movable body from the outside of the movable body, the owner may be able to stop the movable body from the outside. However, if the owner moves away from the movable body and is not monitoring the movable body, the movable body may contact an obstacle.

Therefore, in the future, in a situation where the owner observes a autonomously moving movable body from the outside of the movable body, the owner of the movable body may be legally required to stay within a specified distance from the movable body (e.g., 6 meters).

FIG. 1schematically illustrates a range of 6 meters from a movable body. As illustrated inFIG. 1, it is desired that an owner7of a movable body9stay at a suitable position that is located within 6 meters from the movable body9and that does not hinder the movement of the movable body9. However, it would not be easy for the owner7to instinctively determine the suitable position located within 6 meters from the movable body9that does not hinder the movement of the movable body9(an approximate distance from the movable body).

For example, it may be contemplated that a portable device carried by the owner of a movable body detects the distance from the movable body, and vibrates when the distance falls within or outside the range of 6 meters. However, the distance detected by radio waves may be inaccurate. In addition, based only on the distance, providing a suitable position that does not hinder the movement of the movable body would be difficult.

Patent Documents

SUMMARY OF THE INVENTION

It is a general object of the present invention to provide a projection apparatus for use with a movable body, in which a suitable position for an owner of the movable body to observe the movable body during autonomous movement is presented.

According to an embodiment of the present invention, a projection apparatus for use with a movable body is provided. The projection apparatus includes a parking controller configured to cause the movable body to automatically move to a parking space, and a projector configured to project light onto a road surface in vicinity of the movable body. When the parking controller causes the movable body to automatically park in the parking space, the projector projects the light onto the road surface located within a predetermined distance from the movable body.

According to an embodiment of the present invention, a portable device for communicating with a projection apparatus for use with the movable body is provided. The projection apparatus includes a parking controller configured to cause the movable body to automatically park in a parking space, and a projector configured to project light onto a road surface in vicinity of the movable body. The portable device is configured to receive position information indicating a position located within a predetermined distance from the movable body, from the projection apparatus upon transmitting an instruction requesting the parking controller to start automatic parking to the projection apparatus, and highlight and display the position indicated by the position information in overhead view around the movable body.

According to an embodiment of the present invention, a non-transitory recording medium storing a program for causing a portable device configured to communicate with a projection apparatus for use with the movable body to execute a process is provided. The projection apparatus includes a parking controller configured to cause the movable body to automatically park in a parking space, and a projector configured to project light onto a road surface around the movable body. The process includes receiving position information indicating a position located within a predetermined distance from the movable body, from the projection apparatus upon transmitting an instruction requesting the parking controller to start automatic parking to the projection apparatus, and highlighting and displaying the position indicated by the position information in an overhead view around the movable body

DESCRIPTION OF THE EMBODIMENTS

According to an embodiment of the present invention, a projection apparatus for use with a movable body is provided, in which a suitable position for an owner of the movable body to observe the movable body during autonomous movement is presented.

First Embodiment

First, referring toFIG. 2, an overview of an observation position projected by a projection apparatus for use with the movable body according to a first embodiment will be described.FIG. 2is drawing illustrating an example of projection light projected by the projection apparatus for use with the movable body when viewed from overhead. A movable body9is to be parked in a parking space11(such as a garage) in accordance with an instruction from the owner of the movable body9. The position of the parking space11is identified by a camera or an ultrasonic sensor. Thus, the movable body9is able to predict a driving trajectory to the parking space11. Further, a predetermined distance (such as 6 meters or less, which is hereinafter referred to as a “recommended observation range8”) from the current position of the movable body9can be determined based on a preset reference point, such as the center of the movable body9.

The movable body9estimates an area that is located within the recommended observation range8and that does not overlap a driving trajectory, and project projection light21indicating a recommended observation position to this area. InFIG. 2, the projection light21indicating the recommended observation position is projected to an area on the left front side of the movable body9. Accordingly, the owner of the movable body9can observe the movable body9at the recommended observation position that is located within the recommended observation range8and that does not overlap a driving trajectory, instead of instinctively determining an approximate distance from the movable body9.

Further, the movable body9may also project projection light indicating the entirety of or the outer edge of the recommended observation range8. InFIG. 2, projection light23indicating the outer edge of the recommended observation range8is depicted as a circle centered on the movable body9. This allows the owner of the movable body9to recognize the recommended observation range8itself, and to observe the movable body9from a position within the recommended observation range8during automatic parking. Further, inFIG. 2, projection light22indicating a driving trajectory of the movable body9for automatic parking is also depicted. Accordingly, the owner of the movable body9can predict the moving direction of the movable body9, and move within the recommended observation range8while avoiding the driving trajectory of the movable body9.

Further, the owner of the movable body9carries a portable device2that communicates with the projection apparatus installed in the movable body. Accordingly, the owner of the movable body can start automatic parking from the outside of the vehicle (transmit an instruction to start automatic parking to the vehicle). The portable device2receives position information on a recommended observation position from the projection apparatus for use with the movable body, and displays the recommended observation position on a display of the portable device2.

Terminology

The term “movable body” refers to an object that is movable on the ground by some power. Examples of the movable body include a vehicle and a wheelchair. Preferably, the movable body according to the present embodiment automatically moves itself.

The term “road surface” is a surface on which the movable body travels. In the present embodiment, the road surface includes a place called a parking lot; however, the road surface may include any place where the movable body is movable.

Projection of light refers to projection of an image of light or shadows. Specifically, projection of light means that what is termed as indirect light is made visible.

<Installation Positions of Projectors>

FIG. 3is a drawing illustrating examples of positions of projectors31installed on the movable body9. The projectors31are installed at positions from which at least the projection light21can be projected to a recommended observation position. Depending on the parking situation, a recommended observation position differs within the recommended observation range8. Therefore, the projectors31are installed on the movable body9such that light can be projected to any position within the recommended observation range8. For this reason, it may be effective to install a plurality of projectors31, but one projector31may be installed.

As illustrated inFIG. 3, by installing the projectors31on the four corners of a roof401, projection light can be projected to any position within the recommended observation range8. The projectors31are not required to be installed on the roof401, and may be installed on the four corners402of the vehicle body. Note that the projectors31may be used for purposes other than projecting the projection light21indicating a recommended observation position and the circular-shaped projection light23indicating the recommended observation range8.

For example, the projectors31may be used to project the projection light22indicating a driving trajectory of the movable body9Further, the projectors31may be used to project ranges within which various operating units (mainly doors) are opened or closed. Alternatively, the projectors31may be used to project any characters. In order to project a variety of projection light beams, the projectors31may be installed on the center of a front bumper404, the center of a rear bumper405, a door mirror403, the sides406, and any position on the roof. In the following description, unless otherwise particularly specified, projection light is projected by projector(s)31installed at appropriate positions to project the projection light.

Example Configuration

Next, referring toFIG. 4, the configuration of a projection apparatus1for use with a movable body and the portable device2will be described.FIG. 4is a drawing illustrating an example configuration of the projection apparatus1installed in the movable body9and an example configuration of the portable device2. With reference toFIG. 4, not only projection of projection light for automatic parking, but also projection of projection light for assisting an occupant to enter the movable body9will be described. Further, the projection apparatus1for use with the movable body is a type of an in-vehicle apparatus, and may be referred to as a navigation apparatus, a tuner, a car audio, or a display audio.

The entire projection apparatus1is controlled by a projection controller33. InFIG. 4, one or more projectors31, a parking controller32, a window display controller41, a communication unit34, and a body electronic control unit (ECU)35are connected to the projection controller33.

First, the projection controller33is a device called an information processing apparatus, a computer, or a microcomputer, including a CPU, a RAM, a flash memory, an input/output circuitry, and is installed as a board when put into practical use. The same may apply to other various types of controllers or ECUs.

Each of the projectors31uses a laser or a light emitting diode (LED) to project projection light of a given shape on a road surface. The details will be described with reference toFIG. 5andFIG. 6.

The parking controller32is a device that performs control related to automatic parking. A sensor36, a steering ECU37, an engine ECU38, and a brake ECU39are connected to the parking controller32. The parking controller32detects the parking space11via the sensor36(the parking space11may be specified by the owner of the movable body), estimates a driving trajectory for parking in the parking space11, and controls the steering ECU37so as to control the angle of the steering wheel in accordance with the estimated driving trajectory, while also controlling the engine ECU38to move the movable body9forward or backward. If an obstacle is detected or when the steering wheel is turned, the parking controller32controls the brake ECU39to stop the movable body9.

The sensor36may include a camera capable of capturing a peripheral image, an ultrasonic sensor that detects an obstacle, and radar. However, devices included in the sensor36may differ depending on how the movable body9performs automatic parking. Further, for an electric vehicle or a hybrid vehicle, instead of the engine ECU38, a motor ECU may be used to move the electric vehicle or the hybrid vehicle backward or forward.

The window display controller41displays information on transparent displays42placed on various windows of the movable body9. That is, one or more of the windows of the movable body9have the displays42that are transparent and capable of displaying information. As illustrated inFIG. 4, the transparent displays42include a front display42a, a right front door display42b, a right rear door display42c, a left front door display42d, a left rear door display42e, and a rear display42f, which are connected to the window display controller41. The window display controller41may display any information on the transparent displays42.

In the present embodiment, when an occupant (including the owner of the movable body9) approaches a door to enter the movable body9, a message such as “the door will open” or “3, 2, 1, open!” may be displayed on a corresponding transparent display42of the door that the occupant is approaching. Further, at the time of automatic parking, a message such as “moving xx meters forward” may be displayed on the front display42a, and a message such as “moving xx meters backward” may be displayed on the rear display42f.

The communication unit34is a communication unit that communicates with the portable device2carried by the owner of the movable body9. Communication methods such as frequency and modulation may vary depending on the movable body9. Generic communication methods include Bluetooth (registered trademark) and a wireless LAN. When the movable body9is being parked, the communication unit34transmits a radio wave for searching the portable device2within a predetermined distance. When the portable device2detects the radio wave and transmits a pre-stored ID to the communication unit34, the communication unit34detects the approach of the owner of the movable body9. As a result, the door can be opened.

Further, in the present embodiment, when the owner of the movable body9exits the movable body9, and performs an operation for starting parking on the portable device2, the communication unit34receives the operation for starting parking, and the parking controller32performs the above-described parking control. In addition, the projection controller33controls the projectors31to project the projection light21indicating a recommended observation position. Further, the portable device2may receive, from the owner of the movable body9, an operation for stopping the movable body9while the movable body9is moved in accordance with automatic parking control. Accordingly, even if there is an obstacle that is not detected by the sensor36, the contact of the movable body9with the obstacle can be prevented.

Further, the projection controller33transmits a recommended observation position and the recommended observation range8to the portable device2via the communication unit34(if the recommended observation range8is fixed, the recommended observation range8is not required to be transmitted). Accordingly, the owner of the movable body9can check the recommended observation position and the recommended observation range8on a display54of the portable device2. This may be effective when there is an obstacle in the vicinity of a projector31, and the projector31is unable to project projection light21indicating a recommended observation position onto the road surface, or when the projection light21is projected in a blind spot of the owner of the movable body9.

The body ECU35controls the locking, unlocking, opening, and closing of the doors. An entry detection sensor43is connected to the body ECU35. The entry detection sensor43is a sensor that identifies a door that an occupant is approaching, determines the body size of the occupant who is approaching the door, and detects a surrounding obstacle. By utilizing the entry detection sensor43as a camera for capturing an image of the outside the vehicle from the inside, it is possible to detect a door that an occupant is approaching and the body size of the occupant who is approaching the door. Alternatively, the body size of the occupant may be detected by associating an ID transmitted from the portable device2with the occupant's attributes. Examples of the occupant's attributes include height, weight, gender, and age. By detecting the body size of the occupant, the body ECU35can open the door in accordance with the amount of opening that is not excessively large or small for the occupant to enter.

A surrounding obstacle may be detected by utilizing the entry detection sensor43as a camera (preferably a stereo camera) or an ultrasonic sensor. For example, if the movable body9is being parked in a small parking space11and a door is opened widely, the door may hit an obstacle (such as another vehicle or a wall). Thus, the entry detection sensor43detects a distance to the surrounding obstacle, and performs control that opens the door by a distance less than the detected distance.

The body ECU35sets the distance to a surrounding obstacle as the upper limit, and within the upper limit, determines the amount of opening based on the body size of an occupant. The body ECU35transmits, to the projection controller33, information on which door to open and the amount of opening of the door. In accordance with the amount of opening of the door, the projection controller33projects projection light indicating how much the door may be opened. In addition, when the door is closed, it is preferable to project projection light alerting the occupant that the door is to be closed, or projection light indicating that how much the door may be closed (when the door is not completely closed, and the amount of opening is decreased).

Although not illustrated, an exit detector that detects the exit of the owner from the movable body may also be included. In this case, when the exit detector detects the exit of the owner from the movable body, the parking controller32may start automatic parking (namely, while the owner is in the movable body, the owner performs an operation for starting automatic parking upon detection of the exit of the owner). The exit detector may detect the exit of the owner based on a determination that the portable device2is located outside of the movable body (the portable device2communicates with an external antenna) and the doors are locked.

Next, the portable device2will be described. The portable device2is a terminal device carried by the owner of the movable body. The portable device2may be a device dedicated to communication with the projection apparatus1, or may be a generic information processing device. For example, the portable device2may be a keyless entry key dedicated to communication with the projection apparatus1. The keyless entry key is mainly used to unlock and lock the doors and start the engine. Examples of the generic information processing device include a smart device (a smartwatch, a wearable personal computer (PC), or the like), such as a personal smartphone of the owner. If the portable device2is a generic information processing device, a dedicated application (program) is installed on the projection apparatus1.

The portable device2includes a communication unit51, an operation receiver52, a display controller53, and the display54. The communication unit51communicates with the communication unit34of the projection apparatus1in a wireless or wired manner. For example, the communication unit51transmits an instruction to start automatic parking, an emergency stop, or door opening, and receives position information of a recommended observation position and the recommended observation range8.

The operation receiver52receives various operations performed on the portable device2. The operation receiver52receives input of a hardware keyboard and input of a software keyboard displayed on a touch panel. As a result, the operation receiver52receives an instruction to start automatic parking, an emergency stop, or door opening.

The display controller53highlights a recommended observation position received by the communication unit51, and displays the highlighted recommended observation position in an overhead view around the movable body9. In addition, data indicating projection light, which can be projected by the projection apparatus1, can be received and displayed by the portable device2.

Prior to the description of the projectors31, coordinates used by the movable body will be described.FIG. 5is a drawing illustrating the movable body9viewed from overhead and a coordinate system used by the projection controller33. For example, the projection controller33manages the position of projection light in a two-dimensional coordinate system, with the origin O located at the center of the vehicle. InFIG. 5, the X-axis represents the width direction of the vehicle and the Y-axis represents the longitudinal direction of the vehicle. Projection light is generated as an image of the coordinate system. For example, because the outer edge of the recommended observation range8has a circular shape, the projection controller33creates an image of a circle17with the radius of 6 meters from the origin O. The coordinates of each point constituting the circle17are included in the image data. For example, the coordinates of a point at which the circle17intersects the X-axis are (6, 0). Note that the coordinate system is not required to be the same as the real space, and may be uniformly reduced in size.

If multiple projectors31are installed on the movable body9, a projection range of each of the projectors31is determined beforehand. For example, if a total of four projectors31are installed on the four corners of the roof, an area1is assigned to an upper-right projector31, an area2is assigned to a lower-right projector31, an area3is assigned to an upper-left projector31, and an area4is assigned to a lower-left projector31. Therefore, each of the projectors31may determine a corresponding assigned area in accordance with the coordinates included in image data, and project projection light based only on the assigned area of the image data.

The same applies to the projection light21indicating a recommended observation position15. The projection light21indicating the recommended observation position15is projected by a projector31assigned to an area in which the recommended observation position15is included (the area3inFIG. 5).

A driving trajectory18for automatic parking is estimated by using a known method as described inFIG. 7AandFIG. 7B. As an example, the driving trajectory18is affected by the steering angle, the speed of the vehicle, and the size of the wheels. Among them, the speed of the moving vehicle during automatic parking is low and can be thus regarded as approximately constant, and the size of the wheels is fixed. Therefore, in a simplified model, the driving trajectory18may be determined by the steering angle. Once the steering angle and the steering direction (turning direction) are known, a rotational center19and a rotation radius of the center12of the rear wheel axle are determined. The four corners of the vehicle body relative to the center12of the rear wheel axle are fixed. Thus, the coordinates of the four corners of the vehicle body can be calculated based on the coordinates of the center12of the rear wheel axle. The projection controller33causes the projectors31to project, as a driving trajectory, a trajectory of the coordinates of the four corners of the vehicle body determined in the above manner.

Next, the configuration of a projector31will be described with reference toFIG. 6.FIG. 6is a drawing illustrating an example of the configuration of the projector31. As an example, the projector31includes a light source61, a lens62, a scanning unit63, and a driver unit64. The light source61may be a LED or a laser, and produces mainly visible light. The light source61preferably emits light of each of the three primary colors (RGB), but it is possible to project a recommended observation position in one color. The brightness of the light source61may be fixed. Preferably, the brightness of the light source61may be determined in accordance with the brightness of the surrounding area, which is detected by an illuminance sensor.

The lens62collects light generated by the light source61and forms a predetermined shape of light. The scanning unit63scans projection light over the road surface such that the projection light having a specific shape is projected, as controlled by the driver unit64. For example, the scanning unit63may be a galvanometer mirror, an ultrasonic polarizer, or a digital micromirror device (DMD) used in digital light processing (DLP). These devices function to emit light in an instructed direction. By switching light emission directions in a short period of time, it appears to human eyes that an image has been rendered because of an afterimage.

The driver unit64functions to control the projecting direction of the scanning unit63based on the coordinates of image data. The driver unit64receives image data created by the projection controller33, and controls the direction of light emitted by the scanning unit63so that the light of a specified color is projected to the coordinates indicated by the image data. Accordingly, projection light having the same shape as the image data is projected onto the road surface in the vicinity of the movable body9.

Prior to the description of a recommended observation position, estimation of driving trajectories for automatic parking will described with reference toFIG. 7AandFIG. 7B.FIG. 7Ais a drawing illustrating an example of a parking method selection screen301. The parking method selection screen301is displayed on the display54of the portable device2or a display42of the in-vehicle apparatus before the movable body owner starts automatic parking. The parking method selection screen301includes buttons302through305for selecting “parallel parking on the left side”, “parking in a garage on the left side”, “parallel parking on the right side”, and “parking in a garage on the right side”, respectively. The owner of the movable body9selects a button in accordance with the shape of the parking space11and the position of the movable body9. Preferably, the movable body9stops near (such as immediately next to) the parking space11. Alternatively, instead of the owner specifying a parking space11, the parking controller32may locate a parking space11by using the sensor36.

FIG. 7Bis a drawing illustrating a method for estimating driving trajectories for parking in the parking space11. In the following, an example of parking in a garage on the right side will be described.

(1) When a parking space11is specified, the parking controller32detects the parking space in the specified direction. Namely, if the sensor36is a camera, the camera detects the border of the parking space11. If the sensor36is an ultrasonic sensor, the parking controller32uses ultrasound to detects the entire space of the parking space11.

(2) Next, a turning position310is set on the diagonal upper right of the detected parking space11. The turning position310may be automatically determined relative to the parking space11based on the minimum rotational radius or the like of the movable body9.

(3) The parking control unit32determines a rotational center13used to move the movable body9to the turning position310. For example, a point of intersection between an extension line of the axle passing through the current position and an extension line of the axle at the turning position may be determined as the rotational center13. Therefore, a driving trajectory from the current position to the turning position is an arc centered on the rotational center13. Because a radius of rotation is known, the steering angle can be determined by the parking controller32.

(4) Next, a circular region320in which the movable body9interferes with walls or other vehicles when moving from the turning position310to the parking space11is determined. The circular region320is a circle centered on a point of intersection14between the extension line of the axle at the turning position, extending in the opposite direction to the rotational center13, and a vertical line passing through a predetermined point14aof the parking space11. An optimum position for determining the circular region320(such as a position at 50 cm from the entrance of the parking space11) is preliminarily determined as the predetermined point14a. The radius of the circular region320is a distance from the point of intersection14to the predetermined point14aof the parking space11. However, if the circular region320does not cover the region in which the movable body9interferes with walls or other vehicles, circular region320may be slightly increased.

(5) A driving trajectory from the turning position310to the parking space11follows the arc of a circle centered on the point of intersection14. Because the radius of rotation is known, the steering angle can be determined by the parking controller32.

In the described manner, the driving trajectory18aof the movable body9from the current position to the turning position and the driving trajectory18bfrom the turning position to the parking space11can be estimated.

FIG. 8is drawing illustrating an example of a method for determining the recommended observation position based on driving trajectories and the recommended observation range8. Needless to say, the recommended observation position cannot be located within the four corners of the movable body9when the movable body9moves along the driving trajectory18afrom the current position to the turning position, and within the four corners of the movable body9when the movable body9moves along the driving trajectory18bfrom the turning position to the parking space11(hereinafter referred to as a parking trajectory range16).

The projection controller33determines the recommended observation position15within an area satisfying conditions that the recommended observation position15is located within the recommended observation range8and outside the parking trajectory range16, and that there is no obstacle such as another vehicle. For example, the centroid of an area satisfying the above conditions may be set as a recommended observation position15. By using the centroid, the center of the area satisfying the above conditions can be determined, and a certain distance from driving trajectories can be maintained. However, if the centroid is used, the recommended observation position15tends to be located in the vicinity of the center of the recommended observation range8. Thus, instead of using the centroid, the recommended observation position15may be the outermost position of the recommended observation range8on a line connecting the centroid to the center of the recommended observation range8, or may be the innermost position of the recommended observation range8, which does not overlap the driving trajectories and located on the line connecting the centroid to the center of the recommended observation range8.

The shape of the projection light21indicating the recommended observation position15may be a circular shape, a rectangular shape, or a polygonal shape, or may be a star shape or a footprint shape. The projection light21(such as the radius or the diagonal length of the recommended observation position15) may have a size of several tens of centimeters allowing a person to stand at the position.

Note that the size of the recommended observation position15is not limited to several tens of centimeters allowing a person to stand at the position, and the recommended observation position15may be the entire area satisfying the conditions. In this case, the projectors31project projection light to the entire area satisfying the conditions. As the recommended observation range8is indicated to the owner of the movable body9, the owner of the movable body9may move to any position of the area within the recommended observation range8.

InFIG. 8, three recommended observation positions15are illustrated. This is a case where there are three separated areas satisfying the conditions, and three centers of gravity are found. In such a case, all the three recommended observation positions15may be employed, or the recommended observation position15, closest to the driver's seat, may be employed such that the owner of the movable body9is not required to move by a large distance. Alternatively, the recommended observation position15, closest to the parking space11, may be employed. Because other vehicles do not approach the parking space11in which the movable body9is parked, the owner of the movable body9is unlikely to interfere with other vehicles while observing the movable body9.

The projection controller33creates image data indicating a recommended observation position15(including the coordinates of the center of the recommended observation position15, for example) determined in the above-described manner, and transmits the image data to the projectors31. Accordingly, the projectors31can project the projection light21indicating the recommended observation position15.

<Examples of Projection Light Beams>

Examples of some projection light beams projected by the projection apparatus1for use with the movable body will be described with reference toFIG. 9.FIG. 9is a drawing illustrating examples of projection light beams projected by projectors31when the movable body9is moved during automatic parking. InFIG. 9, in addition to the projection light21indicating the recommended observation position and the projection light23indicating the outer edge of the recommended observation range8, the projection light22indicating a driving trajectory, projection light24indicating a message of “moving 1 meter forward”, projection light25indicating a message of “6 meters”, and projection light26indicating the size of the recommended observation range8are projected. By reading the message of “moving 1 meter forward”, the owner of the movable body9understands that the movable body9is to be moved forward. Further, the message of “6 meters” allows the owner to understand that the projection light23indicating the outer edge of the recommended observation range8may represent a boundary determined by the laws and regulations.

<Projection Light for Assisting Occupant to Enter>

Next, referring toFIG. 10AthroughFIG. 10C, projection light for assisting an occupant to enter the movable body9will be described. Namely, the movable body9has already been parked (the movable body9has been either automatically parked or parked by a driver). Assisting the occupant means that assisting the occupant to enter the movable body9being parked.FIG. 10Aillustrates projection light27indicating how much each of the doors may be opened. In this case, upon detection of the occupant by the entry detection sensor43, the body ECU35determines, as a door to be opened, the door located closest to the occupant. Further, the entry detection sensor43detects a distance to an obstacle and the body size of the occupant. The body ECU35sets the distance to the obstacle as the upper limit, and within the upper limit, determines the amount of opening suitable for the body size of the occupant. The projection controller33creates image data based on the determined door and the amount of opening of the door, and transmits the image data to the projectors31. In this case, the image data may indicate a straight line extending from a predetermined position of the door. Thus, the projectors31may project projection light simply indicating the amount of opening, or may project projection light indicating the angle of the straight line in accordance with the amount of opening.

FIG. 10Bis an example of projection light28indicating how much the rear door may be opened. Depending on the movable body9, the rear door may be a double door (French door) having right and left doors that meet at the center and that can be respectively opened. In this case, the projectors31can project the projection light28indicating how much the right and left doors may be opened. If the rear door is not a double door, projection light can be projected in the same manner as inFIG. 10A.

FIG. 10Calso illustrates projection light29in the case of a rear door, particularly a hatchback-type rear door. Generally, the hatchback-type rear door is fully opened without stopping halfway. Thus, as the projection light29, the projectors31may project light indicating a straight line, parallel to the width direction of the movable body9, to a position a predetermined distance away from the rear end of the movable body9. The position of the road surface onto which the projection light29is projected corresponds to the end of the fully-opened rear door.

<Displaying Information on Transparent Displays>

An occupant can see information from both inside and outside of the vehicle because the transparent displays42are transparent. In general, various types of information are considered to be displayed when the occupant is in the movable body9. For example, immediately after the occupant enters the movable body9, traffic congestion information and weather information may be displayed, for example. Thus, when the owner of the movable body9or the occupant sees the transparent displays42while outside the movable body9, characters and graphics displayed on the transparent displays42may be laterally inverted.

Referring toFIG. 11AthroughFIG. 11C, information displayed on a transparent display42will be described.FIG. 11AthroughFIG. 11Care drawings illustrating an example of a method for generating information displayed on the transparent display42.FIG. 11Ais a drawing illustrating characters420of “ABC” displayed for an occupant located in the movable body9. Therefore, when the owner of the movable body9or an occupant located outside the vehicle sees the characters, the characters are flipped horizontally. Thus, the window display controller41performs a horizontal flipping process when displaying information for the owner of the movable body9or an occupant located outside the movable body9.

FIG. 11Bis a drawing illustrating an example of the horizontal flipping process. The left side ofFIG. 11Bdepicts characters421of “ABC” before the horizontal flipping process, and the right side ofFIG. 11Bdepicts characters422of “ABC” after the horizontal flipping process. When information is displayed for the owner of the movable body9or an occupant located outside the movable body9, the window display controller41displays, on the transparent display42, the information on the right side ofFIG. 11B, which is illustration when viewed from the inside of the movable body9.

FIG. 11Cis a drawing illustrating an example of information displayed on the transparent display42after the horizontal flipping process. InFIG. 11C, characters423of “ABC” can be read from the outside of the vehicle.

Next, referring toFIG. 12AthroughFIG. 12C, some messages displayed on transparent displays42will be described. Note that the transparent displays42are used at the time of automatic parking and to assist an occupant to enter the movable body9.

FIG. 12Ais a drawing illustrating an example of information displayed on the front display42aat the time of automatic parking. Driving trajectories to the parking space11are determined in the process of determining a recommended observation position. Thus, whether or not the movable body9moves forward or backward is also determined. When the movable body9moves forward, the window display controller41displays a message71such as “moving 1 meter forward” on the front display42a.

Further, when the movable body9is turned from the turning position, the window display controller41displays a message such as “moving 3 meters backward” on the rear display42f. It may be difficult for the owner of the movable body9located outside the movable body9to determine whether the movable body9is moving forward or backward. However, by looking at the transparent displays42, the owner readily understand the moving direction of the movable body9.

In addition, when the movable body9is stopped due to an obstacle, a message such as “stopped due to an obstacle” may be displayed. Accordingly, the owner located outside the movable body9understands the reason why the movable body9has stopped.

FIG. 12Bis a drawing illustrating an example of information displayed on the left front door display42dwhen an occupant is assisted in entering the movable body9. InFIG. 12B, a message72indicating “welcome, the door will open after 3 seconds” is displayed. The window display controller41displays the message for a certain period of time, and starts a countdown as illustrated inFIG. 12C.

InFIG. 12C, messages “3”, “2”, “1”, and “open” are displayed in chronological order. When window display controller41indicates, to the projection controller33, that the message “open” is displayed, the projection controller33indicates, to the body ECU35, that the message “open” is displayed. Then, the body ECU35opens the door. Accordingly, the occupant can understand the timing at which the door is opened. During this time, projection light27indicating how much the door may be opened is projected by the projection controller33.

The messages displayed by the window display controller41as illustrated inFIG. 12AthroughFIG. 12Care merely examples, and various types of information for assisting the occupant to enter the movable body9may be displayed.

<Information Displayed by Portable Device>

The projection controller33projects projection light onto the road surface as described above. In addition, it is also effective for the portable device2to display information equivalent to the projection light.

FIG. 13is an example of an automatic parking screen350displayed on the display54of the portable device2. The automatic parking screen350ofFIG. 13displays a birds-eye view of the movable body9viewed from overhead. A vehicle icon351is displayed at the center. In addition, a recommended observation position icon352indicating a recommended observation position and driving trajectory lines353are displayed.

It becomes possible to display the automatic parking screen350by causing the communication unit34to transmit, to the portable device2, coordinates of a recommended observation position15and coordinates of a driving trajectory. The portable device2places the pre-stored vehicle icon351at the center, and displays the recommended observation position icon352on the automatic parking screen350based on the coordinates of the recommended observation position15. Further, the portable device2connects the coordinates of the driving trajectory, and displays the driving trajectory lines353.

By causing the portable device2to display information equivalent to projection light, the owner of the movable body9can understand a recommended observation position even if the projectors31are unable to project projection light, or if projection light is projected in a blind spot of the movable body9and the owner of the movable body9is thus unable to visually recognize the projection light.

Further, the portable device2includes a “start parking” button354and a “stop” button355. The “start parking” button354may be pressed by the owner to start automatic parking. When this button is pressed, the portable device2transmits, to the projection apparatus1, operation information for starting automatic parking. The projection apparatus1starts automatic parking after verifying the safety of the surrounding area. The “stop” button355may be pressed by the owner to stop the movable body9. For example, if the movable body9approaches an obstacle, the owner presses the “stop” button355. When the portable device2transmits, to the projection apparatus1, operation information for stopping the movable body9, the projection apparatus1stops the movable body9. If the “start parking” button354is pressed thereafter, automatic parking is resumed.

Note that the display54of the portable device2may be a touch panel, and may include various types of buttons such as buttons for opening and closing the doors.

Referring toFIG. 14, a process in which the projection apparatus1for use with the movable body projects projection light and displays information on transparent displays42will be described.FIG. 14is a flowchart illustrating an example of a process in which the projection apparatus1projects projection light and displays information on transparent displays42at the time of automatic parking.

First, the parking controller32receives an instruction to start automatic parking (S1). It is assumed that the portable device2receives an instruction to start automatic parking and transmits the received instruction to the movable body9. However, the parking controller32may receive an instruction within the movable body9. After the owner of the movable body9performs an operation for starting automatic parking in the movable body9, the owner may move beyond the movable body9or may stay in the movable body9.

The parking controller32estimates driving trajectories to the parking space11(S2). The details have been described with reference toFIGS. 7A and 7B. If the parking controller32is unable to estimate driving trajectories due to an obstacle, the parking controller32may display a message, such as “please change the parking space11” or “please move the movable body9”, on the display54of the portable device2or a display of the in-vehicle apparatus.

The projection controller33determines a recommended observation position15based on the recommended observation range8and the driving trajectories (S3). The projection controller33transmits coordinates of the recommended observation position15to the portable device2(S4). The projection controller33may transmit coordinates of the recommended observation range8. As the recommended observation range8is fixed, the coordinates of the recommended observation range8are known to the portable device2. Further, the coordinates of the recommended observation position15are not required to be continuously transmitted to the portable device2, and may be transmitted to the portable device2only when an obstacle is detected near the projectors31.

Next, the projection controller33transmits image data indicating the recommended observation position15to the projectors31. The projectors31project the projection light21indicating the recommended observation position15(S5).

Further, the projection controller33determines whether the movable body9moves forward or backward based on a driving trajectory. When the movable body9moves forward, the projection controller33requests the window display controller41to display a predetermined message on the front window. Accordingly, when the projection light21indicating the recommended observation position15is projected at the same time when the message indicating moving forward is displayed on the front window (S6).

Next, the parking controller32causes the movable body9to start moving along the driving trajectory (S7). Before the movable body9starts moving, the parking controller32may request permission from the owner of the movable body9. Further, the position of the recommended observation position15relative to the movable body9is changed as the movable body9moves. For this reason, the coordinates of the recommended observation position15relative to the movable body9are repeatedly calculated in accordance with translation and rotation (change in the angle of yaw) with respect to a reference position such as the center of the movable body9before movement, and the latest image data indicating the recommended observation position15is transmitted to the projectors31. This prevents a change in the position of the projection light21indicating the recommended observation position15. After the movable body9starts moving, the projection of the projection light21may stop.

The parking controller32controls the steering ECU37and the engine ECU38when moving the movable body9along the driving trajectory. Because the radius of rotation for the driving trajectory is known, the movable body9may move at a constant speed with a constant steering angle.

The parking controller32determines whether the movable body9has reached a turning position based on the amount of movement detected from the wheel rotation pulses or the like (S8). The parking controller32continues to move the movable body9until the movable body9reaches the turning position.

When the movable body9has reached the turning position (yes in S8), the moving direction of the movable body9changes. Thus, the projection controller33requests the window display controller41to display a predetermined message on the rear window. Accordingly, the message indicating moving backward is displayed on the rear window (S9).

The parking controller32causes the movable body9to start moving along a driving trajectory estimated for backward movement (S10). Because the steering direction is reversed, the steering ECU37changes the steering angle based on the radius of rotation for the driving trajectory estimated for backward movement.

The parking controller32determines whether the movable body9has reached the parking space11based on the amount of movement detected from the wheel rotation pulses or the like (S11). It is more preferable to use a sensor to determine whether the movable body9has entered the parking space11. The parking controller32continues to move the movable body9until the movable body9reaches the parking space11.

When the movable body9has reached the parking space11(yes in S11), the projection controller33requests the window display controller41to display a predetermined message on the front window. As a result, the message indicating the completion of automatic parking is displayed (S12). Accordingly, the owner of the movable body9can check the completion of automatic parking from the outside of the movable body9.

Next, referring toFIG. 15, projecting projection light and displaying information on transparent displays42when assisting an occupant to enter the movable body9will be described.FIG. 15is a flowchart illustrating an example of a process in which the projection apparatus1for use with the movable body projects projection light and displays information on transparent displays42when assisting an occupant to enter the movable body9.

The communication unit34transmits a radio wave for searching the portable device2within a predetermined distance from the movable body9being parked. The communication unit34determines whether the portable device2has been detected (S21).

When the portable device2has been detected (yes in S21), the body ECU35uses the entry detection sensor43to identify a door that an occupant is approaching (S22).

The entry detection sensor43determines the body size of the occupant (S23). For example, the entry detection sensor43may use a stereo camera to perform three-dimensional measurements and determine the body size, such as large, medium, or small. In this case, even if the occupant carries luggage, the body size of the occupant carrying luggage can be determined. In addition, the body size may be determined based on attribute information associated with an ID transmitted from the portable device2. Further, the entry detection sensor43may identify the occupant using face recognition, and may determine the body size based on the weight of the occupant previously detected by a pressure sensor installed in a vehicle seat.

Next, the entry detection sensor43detects the distance to an obstacle near the door (S24). Preferably, the distance to an obstacle is detected in step S21before the occupant is detected, because the obstacle may be misidentified as the occupant.

Next, the body ECU35sets the distance to the obstacle as the upper limit, and within the upper limit, determines the amount of opening based on the body size (S25). For example, if the body size is large and no obstacle is detected, 50 cm preliminarily associated with the large body size is set as the amount of opening. Further, if the body size is large and the distance to a detected obstacle is 40 cm, 40 cm is set as the amount of opening.

Next, the projection controller33determines whether an instruction to open the door is received from the portable device2(S26). This is to determine whether the occupant desires to enter the vehicle. In addition, this prevents the door from being opened if the occupant is simply moving near the movable body. When the occupant presses a predetermined button on the portable device2, the communication unit51of the portable device2transmits an instruction to open the door to the communication unit34of the projection apparatus1for use with the movable body.

When the instruction to open the door is received (yes in S26), the projection controller33projects projection light indicating how much the door may be opened, in accordance with the amount of opening obtained from the body ECU35(S27).

Then, the projection controller33requests the window display controller41to display a predetermined message on the window of the door that the occupant is approaching. As a result, a message indicating the opening of the door is displayed (S28).

When the projection control unit33indicates, to the body ECU35, that the message has been displayed, the body ECU35opens the door that the occupant is approaching (S29). Accordingly, the timing of opening the door and displaying the message can be synchronized. The timing may be controlled by the body ECU35, instead of controlled by the projection controller33. In this case, when the body ECU35receives the instruction to open the door in step S26, the body ECU35indicates, to the projection controller33, the number of seconds after which the door is to be opened. Further, the projection controller33indicates, to the window display controller41, the number of seconds after which the door is to be opened. The window display controller41displays a message that counts down the number of seconds. Accordingly, the timing of opening the door and displaying the message can be synchronized in this case as well.

Summary

As described above, the projection apparatus1for use with the movable body according to the present embodiment estimates a position that is located within the recommended observation range8and that does not overlap driving trajectories. Further, the projection apparatus1for use with the movable body according to the present embodiment projects the projection light21indicating a recommended observation position to the estimated position. Accordingly, the owner of the movable body9is able to observe the movable body9at the recommended observation position that is located within the recommended observation range8and that does not overlap the driving trajectories, instead of instinctively determining an approximate distance from the movable body9. Further, a message indicating whether the movable body9moves forward or backward is displayed on a transparent display42. Thus, the owner of the movable body9is able to understand the moving direction of the movable body9. Further, when an occupant enters the movable body9, the amount of opening of the door is projected, and a transparent display42displays a message indicating that the door is to be opened.

Second Embodiment

In a second embodiment, a projection apparatus1that uses a voice message to prompt the owner of the movable body to enter the recommended observation range8will be described.

FIG. 16is a drawing illustrating an overview of a voice message transmitted from the projection apparatus1for use with the movable body according to the second embodiment and the positions of the owner of the movable body9. As described in the first embodiment, the recommended observation range8is a predetermined distance from the current position of the movable body9. The projection apparatus1transmits a voice message prompting an owner7of the movable body9to move within the recommended observation range8, and the portable device2outputs the voice message.

(1) For example, when the owner7of the movable body9approaches the recommended observation range8from the outside of the recommended observation range8, the portable device2outputs a voice message that counts down the distance from the owner7to the recommended observation range8, such as “3 meters, 2 meters, 1 meter to the area”. Accordingly, the owner7of the movable body9understands that the owner7is gradually approaching the recommended observation range8, and can figure out how much closer the owner7needs to approach.

(2) When the owner7of the movable body9has entered the recommended observation range8, the portable device2outputs a voice message such as “you have entered the recommended area. Please stay at the current position until automatic parking is completed.” Accordingly, the owner7of the movable body9understands that the owner7can stay at the current position.

(3) If the owner7of the movable body9moves beyond the recommended observation range8after entering the recommended observation range8, the portable device2outputs a voice message such as “you have moved beyond the recommended area. Please enter the area again.” Accordingly, the owner7of the movable body9understands that the owner7has moved beyond the recommended observation range8and needs to approach the movable body to enter the recommended observation range8again.

As described above, the projection apparatus1according to the present embodiment causes the portable device2to output a voice message prompting the owner7of the movable body9to move within the recommended observation range8, in addition to projecting projection light and causing the portable device2to display an overhead view. Accordingly, the owner7of the movable body9can more securely observe the movable body9during automatic parking at a position within the recommended observation range8.

Configuration Example

FIG. 17is a drawing illustrating an example configuration of the projection apparatus1installed in the movable body9and an example configuration of the portable device2. InFIG. 17, the elements denoted by the same reference numerals as the elements ofFIG. 4have similar functions. Thus, only the main elements of the present embodiment may be described.

The projection apparatus1according to the present embodiment further includes a voice message generator45and a distance detector44. First, the distance detector44repeatedly detects a distance to the owner of the movable body9. For example, a distance to the owner of the movable body9may be detected based on the strength of a radio wave received by the communication unit34from the portable device2. The communication unit51of the portable device2transmits the strength of a radio wave at the time of output to the projection apparatus1. Thus, the distance detector44compares the strength of a received radio wave to the strength of a radio wave at the time of output, and coverts to a distance. Alternatively, the distance detector44may use a stereo camera to detect a distance to the owner of the movable body9, or may use a laser, light detection and ranging (LIDAR), or an ultrasonic sensor to detect a distance to the owner of the movable body9.

The voice message generator45generates a voice message based on the distance detected by the distance detector44, based on whether the owner of the movable body9is located inside or outside the recommended observation range8, and based on the moving direction of the owner the movable body9. The details will be described below. When the communication unit51of the portable device2receives the voice message, the voice message is output from a speaker55in real time.

FIG. 18is flowchart illustrating an example of a process in which the portable device2outputs a voice message generated by the voice message generator45. The process ofFIG. 18starts when automatic parking is started.

First, the distance detector44detects a distance to the owner of the movable body9(S31). The distance is measured repeatedly or periodically.

The voice message generator45determines whether the owner of the movable body9is located outside the recommended observation range8, based on the distance to the owner of the movable body9(S32).

When it is determined as “no” in step S32, the owner of the movable body9is located within the recommended observation range8. Thus, the voice message generator45generates a voice message prompting the owner to stay within the recommended observation range8(S36). The voice message is transmitted to the portable device2, and the portable device2outputs the voice message, such as “you have entered the recommended area. Please stay at the current position until automatic parking is completed.” When the owner of the movable body9has entered the recommended observation range8once, a flag A is turned on. While the flag A is turned on, the voice message of step S36is not generated.

When it is determined as “yes” in step S32, the voice message generator45turns the flag A off, and determines whether the owner of the movable body9has ever entered the recommended observation range8after the start of the automatic parking (S33). The voice message generator45determines whether the owner has ever entered the recommended observation range8before based on the flag A of step S36.

When it is determined as “no” in step S33, the voice message generator45generates a voice message indicating the distance to the recommended observation range8(S34). The voice message is transmitted to the portable device2, and the portable device2outputs the voice message, such as “3 meters to the area”. As the distance to the owner of the movable body9decreases, voice messages such as “2 meters to the area” and “1 meter to the area” can be output.

When it is determined as “yes” in step S33, the voice message generator45generates a voice message prompting the owner to enter the recommended observation range8again (S35). The voice message is transmitted to the portable device2, and the portable device2outputs the voice message, such as “you have moved beyond the recommended area. Please enter the area again.”

As describe above, the projection apparatus1according to the present embodiment causes the portable device2to output a voice message prompting the owner of the movable body9to move within the recommended observation range8.

InFIG. 17, the projection apparatus1includes the voice message generator45and the distance detector44. However, the portable device2may include the voice message generator45and the distance detector44. In this case, the portable device2generates a voice message and outputs the voice message from the speaker55.

If the portable device2generates a voice message, transmitting and receiving the voice message is not required. If the projection apparatus1generates a voice message, projecting projection light and displaying the voice message can be readily synchronized. For example, when the owner of the movable body9enters the recommended observation range8, it is possible to project the projection light23indicating the outer edge of the recommended observation range8while also outputting a voice message.

Other Application Examples

Although the embodiments of the present invention have been described above, the present invention is not limited the above-described embodiment. Various variations and modifications may be made without departing from the scope of the present invention.

For example, the occupant detecting sensor has been described as a camera installed in the vehicle; however, the occupant detecting sensor may be a camera installed at the entrance of an occupant's house. In this case, the camera installed at the entrance communicates with the projection apparatus1, and indicates, to the projection apparatus1, the body size of the occupant before entering the vehicle.

Further, the recommended observation range8is not required to have a circular shape. In addition, instead of the predetermined distance (6 m) from the center of the movable body9, a predetermined distance from the outer edge of the recommended observation range8may be used as the recommended observation range8.

Further, information such as “the trunk will open” may be displayed on the transparent display of the rear door.

Further, in the above-described embodiments, a recommended observation position at the time of parking has been described. However, in other situations such as a vehicle leaves from a garage and the owner of the vehicle is not located within the vehicle, a recommended observation position at which the owner observes the vehicle may be used.

Further, in the above-described embodiments, the projection apparatus1for use with the movable body calculates driving trajectories for parking, and determines a recommended observation position. However, a server may determine driving trajectories and a recommended observation position, and may transmit the driving trajectories and the recommended observation position to the projection apparatus1for use with the movable body. In this case, surrounding information obtained by the sensor36is transmitted from the projection apparatus1for use with the movable body transmits to the server, and one or both of the driving trajectories and the recommended observation position are transmitted to the projection apparatus1for use with the movable body.