System and method for social networking using an augmented reality display

A method for social networking using a multi-focal plane augmented reality display of a host vehicle includes receiving social-networking data from a remote device. The social-networking data includes information about at least one social interest of a remote user of the remote device. The remote device is located within a viewable area of a vehicle user of the host vehicle. The method further includes determining whether at least one social interest of the remote user matches a vehicle-user social interest of the vehicle user of the host vehicle using the social-networking data. The method further includes transmitting a command signal to the multi-focal plane augmented reality display of the host vehicle to display a virtual image on the multi-focal plane augmented reality display. The virtual image is indicative of the vehicle.

INTRODUCTION

The present disclosure relates to a system and method for social networking using a multi-focal plane augmented reality display.

This introduction generally presents the context of the disclosure. Work of the presently named inventors, to the extent it is described in this introduction, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against this disclosure.

Some vehicles include displays to provide information to a vehicle user. However, these displays do not necessarily present information relating to social networking. In the present disclosure, the term “social networking” means the use of internet-based social media platforms to stay connected with friends, family, acquaintances, or peers or to find new connections having common social interests. Providing social networking information may help the vehicle user connect with other people that have similar interests. In the present disclosure, the term “social networking information” means information about social networking. It is desirable to develop a system and a method for social networking using a multi-focal plane augmented reality display of a vehicle based on real-time vehicle positioning.

SUMMARY

The present disclosure describes a system and method for presenting interest-based social networking information to the vehicle user on an augmented reality display, such as a head-up display. Based on the personal settings of the vehicle user, the augmented reality display presents the vehicle user with the location of people with similar interests or attending local events. Contact information may be presented on a vehicle display.

In an aspect of the present disclosure, a method for social networking using a multi-focal plane augmented reality display of a host vehicle includes receiving social-networking data from a remote device. The social-networking data includes information about at least one social interest of a remote user of the remote device. The remote device is located within a viewable area of a vehicle user of the host vehicle. The viewable area is an area that is visible by the vehicle user through a windshield of the host vehicle, in front of the host vehicle, and within a predetermined viewable distance from the host vehicle. The method further includes determining whether at least one social interest of the remote user matches a vehicle-user social interest of the vehicle user of the host vehicle using the social-networking data. Further, the method includes, in response to determining that at least one social interest of the remote user matches the vehicle-user social interest of the vehicle user, transmitting a command signal to the multi-focal plane augmented reality display of the host vehicle to display a virtual image on the multi-focal plane augmented reality display. The virtual image is indicative of the vehicle-user social interest that matches at least one social interest of the remote user. The method described in this paragraph improves vehicle technology by presenting interest-based social networking information to the vehicle user on an augmented reality display, such as a head-up display, thereby allowing the vehicle user to locate of people with similar interests or attending the same local events.

In an aspect of the present disclosure, the method further includes determining a location of the host vehicle relative to the remote user having at least one social interest that matches the vehicle-user social interest.

In an aspect of the present disclosure, the method further includes determining a type of the virtual image to be displayed by the multi-focal plane augmented reality display based on at least one social interest of the remote user that matches the vehicle-user social interest.

In an aspect of the present disclosure, the method includes determining a location of eyes of the vehicle user of the host vehicle and determining a location of the virtual image based on the location of the eyes of the vehicle user. Transmitting the command signal to the multi-focal plane augmented reality display includes commanding the multi-focal plane augmented reality display of the host vehicle to display the virtual image at the location determined based on the location of the eyes of the vehicle user.

In an aspect of the present disclosure, the method includes determining a location of eyes of the vehicle user of the host vehicle, determining a location of a head of the vehicle user of the host vehicle, and determining a location of the virtual image based on the location of the eyes of the vehicle user and the location of the head of the host vehicle. Transmitting the command signal to the multi-focal plane augmented reality display includes commanding the multi-focal plane augmented reality display of the host vehicle to display the virtual image at the location determined based on the location of the eyes and the location of the head of the vehicle user.

In an aspect of the present disclosure, the multi-focal plane augmented reality display is configured as a head-up display, and the virtual image is displayed on the windshield of the host vehicle.

In an aspect of the present disclosure, the host vehicle includes an information display. The method further includes displaying contact information for the remote user with at least one social interest of the remote user that matches the vehicle-user social interest.

In an aspect of the present disclosure, the method further includes, after transmitting the command signal to the multi-focal plane augmented reality display of the host vehicle to display the virtual image on the multi-focal plane augmented reality display, determining whether the remote user with at least one social interest that matches the vehicle-user social interest is still within the viewable area of the vehicle user.

In an aspect of the present disclosure, determining whether the remote user with at least one social interest that matches the vehicle-user social interest is still within the viewable area of the vehicle user includes: determining whether the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is within the predetermined viewable distance from the host vehicle; and determining whether the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is in the area in front of the host vehicle.

In an aspect of the present disclosure, the method further includes, in response to determining that the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is not within the predetermined viewable distance from the host vehicle or that the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is not in the area in front of the host vehicle, determining that the remote user with at least one social interest that matches the vehicle-user social interest is not within the viewable area of the vehicle user. The method further includes, in response to determining that the remote user with at least one social interest that matches the vehicle-user social interest is not within the viewable area of the vehicle user, transmitting an off signal to the multi-focal plane augmented reality display to discontinue showing the virtual image on the multi-focal plane augmented reality display.

The present disclosure also describes a system for social networking in a of a host vehicle. In an aspect of the present disclosure, the system includes a transceiver configured to receive social-networking data from a remote device. The social-networking data includes information about at least one social interest of a remote user of the remote device. The remote device is located in a viewable area of a vehicle user of the host vehicle, and the viewable area is an area that is visible by the vehicle user through a windshield of the host vehicle, in front of the host vehicle, and within a predetermined viewable distance from the host vehicle. The system further includes a multi-focal plane augmented reality display configured to display a virtual image and a controller in communication with the transceiver and the multi-focal plane augmented reality display. The controller is configured to: receive the social-networking data from the remote device; determine whether at least one social interest matches a vehicle-user social interest of the vehicle user of the host vehicle; and in response to determining that at least one social interest of the remote device matches the vehicle-user social interest of the vehicle user, transmit a command signal to the multi-focal plane augmented reality display of the host vehicle to display a virtual image on the multi-focal plane augmented reality display. The virtual image is indicative of the vehicle-user social interest of the remote user that matches at least one social interest of the remote device. The system described in this paragraph improves vehicle technology by presenting interest-based social networking information to the vehicle user on an augmented reality display, such as a head-up display, thereby allowing the vehicle user to locate of people with similar interests or attending the same local events.

In an aspect of the present disclosure, the system further includes a plurality of sensors in communication with the controller. The controller is further configured to determine a location of the host vehicle relative to the remote device having at least one social interest that matches the vehicle-user social interest based on at least one signal received from the plurality of sensors.

In an aspect of the present disclosure, the controller is configured to determine a type of the virtual image to be displayed by the multi-focal plane augmented reality display based on at least one social interest that matches the vehicle-user social interest.

In an aspect of the present disclosure, the plurality of sensors includes a user tracker in communication with the controller. The controller is configured to determine a location of eyes of the vehicle user of the host vehicle based on an input from the user tracker and determine a location of the virtual image based on the location of the eyes of the vehicle user. The controller is further configured to command the multi-focal plane augmented reality display of the host vehicle to display the virtual image at the location determined based on the location of the eyes of the vehicle user.

In an aspect of the present disclosure, the sensors include a user tracker in communication with the controller. The controller is configured to: determine a location of eyes of the vehicle user of the host vehicle based on an input from the user tracker; determine a location of a head of the vehicle user of the host vehicle based on the input from the user tracker; determine a location of the virtual image based on the location of the eyes of the vehicle user and the location of the head of the host vehicle; and command the multi-focal plane augmented reality display of the host vehicle to display the virtual image at the location determined based on the location of the eyes and the location of the head of the vehicle user.

In an aspect of the present disclosure, the multi-focal plane augmented reality display is configured as a head-up display. The controller is configured to command the multi-focal plane augmented reality display to display the virtual image on the windshield of the host vehicle.

In an aspect of the present disclosure, the host vehicle includes an information display in communication with the controller. The controller is further configured to command the information display to display contact information for the remote user with at least one social interest that matches the vehicle-user social interest.

In an aspect of the present disclosure, the controller is configured to determine whether the remote user with at least one social interest that matches the vehicle-user social interest is still within the viewable area of the vehicle user after transmitting the command signal to the multi-focal plane augmented reality display of the host vehicle to display the virtual image on the multi-focal plane augmented reality display.

In an aspect of the present disclosure, the controller is configured to determine whether the remote user with at least one social interest that matches the vehicle-user social interest is still within the viewable area of the vehicle user by: determining whether the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is within the predetermined viewable distance from the host vehicle; and determining whether the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is in the area in front of the host vehicle.

In an aspect of the present disclosure, the controller is configured to, in response to determining that the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is not within the predetermined viewable distance from the host vehicle or that the remote device of the remote user with at least one social interest that matches the vehicle-user social interest is not in the area in front of the host vehicle, determine that the remote user with at least one social interest that matches the vehicle-user social interest is not within the viewable area of the vehicle user. The controller is configured to, in response to determining that the remote user with at least one social interest that matches the vehicle-user social interest is not within the viewable area of the vehicle user, transmit an off signal to the multi-focal plane augmented reality display to discontinue showing the virtual image on the multi-focal plane augmented reality display.

Further areas of applicability of the present disclosure will become apparent from the detailed description provided below. It should be understood that the detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.

The above features and advantages, and other features and advantages, of the presently disclosed system and method are readily apparent from the detailed description, including the claims, and exemplary embodiments when taken in connection with the accompanying drawings.

DETAILED DESCRIPTION

Reference will now be made in detail to several examples of the disclosure that are illustrated in accompanying drawings. Whenever possible, the same or similar reference numerals are used in the drawings and the description to refer to the same or like parts or steps.

With reference toFIG.1, a host vehicle10generally includes a chassis12, a body14, front and rear wheels17and may be referred to as a vehicle system. In the depicted embodiment, the host vehicle10includes two front wheels17aand two rear wheels17b. The body14is arranged on the chassis12and substantially encloses components of the host vehicle10. The body14and the chassis12may jointly form a frame. The wheels17are each rotationally coupled to the chassis12near a respective corner of the body14. The host vehicle10includes a front axle19coupled to the front wheels17aand a rear axle25coupled to the rear wheels17b.

In various embodiments, the host vehicle10may be an autonomous vehicle and a control system98is incorporated into the host vehicle10. The control system98may be referred to as the system or the system for social networking using one or more displays29, such as a multi-focal plane augmented reality display. The host vehicle10is, for example, a vehicle that is automatically controlled to carry passengers from one location to another. The host vehicle10is depicted in the illustrated embodiment as a pickup truck, but it should be appreciated that other vehicles including, trucks, sedans, coupes, sport utility vehicles (SUVs), recreational vehicles (RVs), etc., may also be used. In an embodiment, the host vehicle10may be a so-called a Level Two, a Level Three, Level Four, or Level Five automation system. A Level Four system indicates “high automation,” referring to the driving mode-specific performance by an automated driving system of aspects of the dynamic driving task, even if a human driver does not respond appropriately to a request to intervene. A Level Five system indicates “full automation,” referring to the full-time performance by an automated driving system of aspects of the dynamic driving task under a number of roadway and environmental conditions that can be managed by a human driver. In Level3vehicles, the vehicle systems perform the entire dynamic driving task (DDT) within the area that it is designed to do so. The vehicle operator is only expected to be responsible for the DDT-fallback when the host vehicle10essentially “asks” the driver to take over if something goes wrong or the vehicle is about to leave the zone where it is able to operate. In Level2vehicles, systems provide steering, brake/acceleration support, lane centering, and adaptive cruise control. However, even if these systems are activated, the vehicle operator at the wheel must be driving and constantly supervising the automated features.

As shown, the host vehicle10generally includes a propulsion system20, a transmission system22, a steering system24, a brake system26, a sensor system28, an actuator system30, at least one data storage device32, at least one controller34, and a communication system36. The propulsion system20may, in various embodiments, include an electric machine such as a traction motor and/or a fuel cell propulsion system. The host vehicle10may further include a battery (or battery pack)21electrically connected to the propulsion system20. Accordingly, the battery21is configured to store electrical energy and to provide electrical energy to the propulsion system20. In certain embodiments, the propulsion system20may include an internal combustion engine. The transmission system22is configured to transmit power from the propulsion system20to the vehicle wheels17according to selectable speed ratios. According to various embodiments, the transmission system22may include a step-ratio automatic transmission, a continuously-variable transmission, or other appropriate transmission. The brake system26is configured to provide braking torque to the vehicle wheels17. The brake system26may, in various embodiments, include friction brakes, brake by wire, a regenerative braking system such as an electric machine, and/or other appropriate braking systems. The steering system24influences a position of the vehicle wheels17and may include a steering wheel33. While depicted as including a steering wheel33for illustrative purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system24may not include a steering wheel33.

The sensor system28includes one or more sensors40(i.e., sensing devices) that sense observable conditions of the exterior environment and/or the interior environment of the host vehicle10. The sensors40are in communication with the controller34and may include, but are not limited to, one or more radars, one or more light detection and ranging (lidar) sensors, one or more proximity sensors, one or more odometers, one or more ground penetrating radar (GPR) sensors, one or more steering angle sensors, one or more global positioning systems (GPS) transceivers45, one or more tire pressure sensors, one or more cameras41(e.g., optical cameras and/or infrared cameras), one or more gyroscopes, one or more accelerometers, one or more inclinometers, one or more speed sensors, one or more ultrasonic sensors, one or more inertial measurement units (IMUs) and/or other sensors. Each sensor40is configured to generate a signal that is indicative of the sensed observable conditions of the exterior environment and/or the interior environment of the host vehicle10. Because the sensor system28provides data to the controller34, the sensor system28and its sensors40are considered sources of information (or simply sources).

The sensor system28includes one or more Global Navigation Satellite System (GNSS) transceivers45(e.g., Global Positioning System (GPS) transceivers) configured to detect and monitor the route data (i.e., route information). The GNSS transceiver45is configured to communicate with a GNSS to locate the position of the host vehicle10in the globe. The GNSS transceiver45is in electronic communication with the controller34.

The actuator system30includes one or more actuator devices42that control one or more vehicle features such as, but not limited to, the propulsion system20, the transmission system22, the steering system24, and the brake system26. In various embodiments, the vehicle features may further include interior and/or exterior vehicle features such as, but are not limited to, doors, a trunk, and cabin features such as air, music, lighting, etc.

The data storage device32stores data for use in automatically controlling the host vehicle10. In various embodiments, the data storage device32stores defined maps of the navigable environment. In various embodiments, the defined maps may be predefined by and obtained from a remote system. For example, the defined maps may be assembled by the remote system and communicated to the host vehicle10(wirelessly and/or in a wired manner) and stored in the data storage device32. The data storage device32may be part of the controller34, separate from the controller34, or part of the controller34and part of a separate system.

The host vehicle10may further include one or more airbags35in communication with the controller34or another controller of the host vehicle10. The airbag35includes an inflatable bladder and is configured to transition between a stowed configuration and a deployed configuration to cushion the effects of an external force applied to the host vehicle10. The sensors40may include an airbag sensor, such as an IMU, configured to detect an external force and generate a signal indicative of the magnitude of such external force. The controller34is configured to command the airbag35to deploy based on the signal from one or more sensors40, such as the airbag sensor. Accordingly, the controller34is configured to determine when the airbag35has been deployed.

The controller34includes at least one processor44and a non-transitory computer readable storage device or media46. The processor44may be a custom made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller34, a semiconductor-based microprocessor (in the form of a microchip or chip set), a macroprocessor, a combination thereof, or generally a device for executing instructions. The computer readable storage device or media46may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor44is powered down. The computer-readable storage device or media46may be implemented using a number of memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or another electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions, used by the controller34in controlling the host vehicle10. The controller34of the host vehicle10may be referred to as a vehicle controller and may be programmed to execute a method100(FIG.7) as described in detail below.

In various embodiments, one or more instructions of the controller34are embodied in the control system98. The host vehicle10includes a user interface23, which may be a touchscreen in the dashboard. The user interface23may include, but is not limited to, an alarm, such as one or more speakers27to provide an audible sound, haptic feedback in a vehicle seat or other object, one or more displays29, one or more microphones31and/or other devices suitable to provide a notification to the vehicle user of the host vehicle10. The user interface23is in electronic communication with the controller34and is configured to receive inputs by a user (e.g., a vehicle operator or a vehicle passenger). For example, the user interface23may include a touch screen and/or buttons configured to receive inputs from a vehicle user11(FIG.6). Accordingly, the controller34is configured to receive inputs from the user via the user interface23.

The host vehicle10may include one or more displays29configured to display information to the user (e.g., vehicle operator or passenger) and may be an augmented reality (AR) display. In the present disclosure, the term “AR display” means a display that presents information to users, while still allowing sight of the outside world. In certain embodiments, the display29may be configured as a head-up display (HUD), and/or an information cluster display. Accordingly, the display29may be an AR HUD or an AR information cluster display. In an AR HUD, images are projected on a windshield39(FIG.6) of the host vehicle10. In an AR information cluster display, the information cluster panel of the host vehicle10includes a screen that presents an image of the front of the host vehicle10captured by one or more cameras41(i.e., a forward image) and virtual images presented on that forward image. As discussed below, the display29may be a multi-focal plane AR display to facilitate manipulation (e.g., size, location, and type) of the virtual images. In certain embodiments, the host vehicle10includes a multi-focal plane augmented reality display29a(FIG.2) configured as a head-up display and an information display29b(FIG.3) configured as a center display.

The communication system36is in communication with the controller34and is configured to wirelessly communicate information to and from other remote devices48, such as but not limited to, other vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems at a remote call center (e.g., ON-STAR by GENERAL MOTORS) and/or personal electronic devices, such as a mobile phone. In the present disclosure, the term “remote device” means a device configured to transmit one or more signals to the host vehicle10while not physically connected to the host vehicle10. In certain embodiments, the communication system36is a wireless communication system configured to communicate via a wireless local area network (WLAN) using IEEE 802.11 standards or by using cellular data communication. However, additional or alternate communication methods, such as a dedicated short-range communications (DSRC) channel, are also considered within the scope of the present disclosure. DSRC channels refer to one-way or two-way short-range to medium-range wireless communication channels specifically designed for automotive use and a corresponding set of protocols and standards. Accordingly, the communication system36may include one or more antennas and/or communication transceivers37for receiving and/or transmitting signals, such as cooperative sensing messages (CSMs). The communication transceivers37may be considered sensors40. The communication system36is configured to wirelessly communicate information between the host vehicle10and another vehicle. Further, the communication system36is configured to wirelessly communicate information between the host vehicle10and infrastructure or other vehicles.

With reference toFIGS.1,2, and3, the system98is configured to command the display29, such as an AR HUD, an AR information cluster display and/or an information display, to present social networking information received from data received from other remote devices48through the communication system36(e.g., V2V communications), and/or eye-tracking data. In doing so, the display29shows the social networking information in a contextual manner by augmenting the road scene with conformal graphics. In the present disclosure, the term “conformal graphic” means synthetic generated content (i.e., a virtual image50) presented as part of the outside world. Accordingly, the display29is a conformal display. In the present disclosure, the term “conformal display” means a display that has the ability to represent synthetically generated content (i.e., one or more virtual images50) as part of the outside world.

InFIG.2, the multi-focal plane augmented reality display29ais configured as a head-up display (HUD) and presents virtual images50indicative of a social interest of remote users49. In the present disclosure, the term “remote user” means a human that is located outside the host vehicle10and that is in possession of one or more remote devices48. In the depicted example, the multi-focal plane augmented reality display29ashows a first virtual image50aadjacent to a first remote user49awith a first social interest, such as attending a local event (e.g., DETROIT TIGERS game), a second virtual image50badjacent to a second remote user49bwith a second social interest (e.g., HARVARD alumni), and a third virtual image50cadjacent to a third remote user49cwith a third social interest (e.g., PINK FLOYD fan). While the second remote user49band the third remote user49care operating a motor vehicle, the first remote user49ais riding a bicycle. It is envisioned, however, that the remote user49may be operating a vehicle or walking. Each virtual image50includes text, logo, slogan (e.g., DETROIT TIGERS logo, PINK FLOYD brand, and/or HARVARD logo), or other information indicative of the social interest of the particular remote user49. As discussed above, the virtual images50may be displayed on an information cluster display instead of (or in addition to) the multi-focal plane augmented reality display29a. Regardless of the type of display29used to show the virtual images50, the display29presents virtual images50based on social-networking data received from remote devices located within a viewable area of a vehicle user11(FIG.6) of the host vehicle10. In the present disclosure, the term “viewable area” means an area that is: (1) visible by the vehicle user11(FIG.6) through a windshield39(FIG.6) of the host vehicle10(FIG.1); (2) in front of the host vehicle10, and (3) within a predetermined viewable distance from the host vehicle10. The predetermined viewable distance may be determined by testing the host vehicle10. In some embodiments, the predetermined viewable distance is twenty feet from the host vehicle10to provide the maximum information possible to a vehicle user11with 20/20 vision. The multi-focal plane augmented reality display29amay solely display virtual images50that are indicative of social interest of the remote users49that match the social interests of the vehicle user11of the host vehicle10. The vehicle user11may input his or her social interests into the controller34through the user interface23before operating the host vehicle10.

InFIG.3, the information display29bis configured as a center information display and presents contact information76associated with each remote user49identified in the multi-focal plane augmented reality display29aas having the same social interest as the vehicle user11. To this end, the display29bshows remote-user information75and the contact information76of the particular remote user49next to the remote-user information75. The contact information76may be the page of the remote user49in a particular Internet-based social media platform, such as FACEBOOK, TWITTER, INSTAGRAM, WHATSAPP, and/or TIKTOK. Instead or in addition to displaying the contact information76of the remote user49, the display29may show a symbol that functions as a link to the page of the remote user49. This symbol may be indicative of one or more Internet-based social media platforms, such as FACEBOOK, TWITTER, INSTAGRAM, WHATSAPP, and/or TIKTOK. The remote-user information75may include text identifying the remote user49having the same social interest as the vehicle user11. For example, the remote-user information75may include text, such as “PINK FLOYD fan”, “TIGER fan heading to the DODGER Stadium” and/or “HARVARD alumni '09”.

With reference toFIGS.4and5, the display29may be a multi-focal plane AR display as mentioned above. In this case, the display29has a first image plane58and a second image plane60. The first image plane58shows the view of the outside world, and the second image plane60is reserved for displaying the virtual images50(FIG.2). The second image plane60spans multiple lanes and the virtual images50appear at a location farther on the roadway surface62relative to the first image plane58. For instance, as shown inFIG.5, the second image plane60covers the left lane52, the central lane54, and the right lane56. As a non-limiting example, in the central lane54, the second image plane60starts at a first predetermined distance D1(e.g., twenty-five meters) from the host vehicle10and ends at a second predetermined distance D2(e.g., ninety meters) from the host vehicle10. Regardless of the specific distances, the second predetermined distance D2is greater than the first predetermined distance D1to help the vehicle user11(FIG.6) see the virtual images50(FIG.3). In the left lane52and the right lane56, the second image plane60is delimited by a sloped boundary that starts at the first predetermined distance D1from the host vehicle10and ends at a third predetermined distance D3(e.g., fifty meters) from the host vehicle10. The third predetermined distance D3is greater than the first predetermined distance D1and less than the second predetermined distance D2to help the vehicle user11(FIG.6) see the virtual images50(FIG.3). As used herein, the term “multi-focal plane AR display” means an AR display that presents images in more than one image plane, wherein the image planes are at different locations. It is desirable to use a multi-focal plane AR display in the presently disclosed system98to easily change the size, type, and/or location of the virtual images50with respect to the view of the outside world.

With reference toFIG.6, the system98includes a user tracker43(e.g., eye tracker and/or head tracker) configured to track the position and movement of the eyes66or the head69of the vehicle user11. In the depicted embodiment, the user tracker may be configured as one or more of cameras41of the host vehicle10. As discussed above, the cameras41are considered sensors40of the host vehicle10. As a sensor40, the user tracker43is in communication with the controller34, which includes a system manager68. During operation of the system98, the system manager68receives at least a first input70and a second input72. The first input70is indicative of the position of the vehicle in space (i.e., the vehicle location in space), and the second input72is indicative of the vehicle user position in the host vehicle10(e.g., the position of the eyes and/or head of the user in the host vehicle10). The first input70may include data such as GNSS data (e.g., GPS data), vehicle speed roadway curvature, and vehicle steering, and this data may be collected from the sensors40of the host vehicle10and/or other remote devices48through the communication system36of the host vehicle10. The second input72may be received from the user tracker (e.g., eye tracker and/or head tracker). The system manager68is configured to determine (e.g., compute) the type, size, shape, and color of the conformal graphics (i.e., virtual images50) based on the first input70(i.e., the vehicle location in space), the second input72(e.g., the position of the eyes and/or head of the user in the host vehicle10), and the sensed vehicle driving environment (which may be obtained through the sensors40). The type, size, shape, and color of the conformal graphics of the virtual image50may be collectively referred to as the virtual image characteristics.

With continued reference toFIG.6, the system98further includes an image engine74, which is part of the display29, and may be an integrated circuit configured to generate the virtual images50. These generated virtual images50are then projected on the windshield39(if the display29is a HUD) to show the virtual images50on the second image plane60along the roadway surface62.

FIG.7is a flowchart of the method100for social networking using the display29, such as the multi-focal plane augmented reality display29aand/or the information display29b. The method100begins at block102, in which the controller34determines that the host vehicle10is being driven using, for example, signals generated by the sensors40. For example, the controller34may receive data from one of the sensors40, such as a speed sensor, to determine that the host vehicle10is moving. Then, the method100proceeds to block104.

At block104, the controller34receives social-networking data. As discussed above, the term “social networking” means the use of internet-based social media platforms to stay connected with friends, family, acquaintances, or peers or to find new connections having, for example, common social interests. The term “social-networking data” means data about social-networking and the remote users49engaged in social networking. In certain embodiments, the social-networking data includes information about at least one social interest of remote users49within the viewable area of the vehicle user11(FIG.6) of the host vehicle10as well as data about the location, movements, trajectory, and/or destination of the remote users49engaged in social networking. As discussed above, the term “viewable area” means an area that is: (1) visible by the vehicle user11(FIG.6) through a windshield39of the host vehicle10(FIG.1); (2) in front of the host vehicle10, and (3) within a predetermined viewable distance from the host vehicle10. To obtain the social-networking data, the controller34may receive at least one signal from one or more sensors40, such as cameras41, GNSS transceivers45(e.g., Global Positioning System (GPS) transceivers), data from the Internet, roadway databases and/or data from the other remote devices48, such as remote vehicles or remote mobile phones, via the communication transceivers37using, for example, V2V communication. Because the remote users49are in possession of remote devices48, the controller34may determine the location, movements, and social interests of the remote users49within the viewable area of the vehicle user11of the host vehicle10using, for example, the social-networking data received from the GNSS transceiver45, another sensor40, or another suitable source of information, such as the remote devices48. At block104, the controller34may also determine the location of the host vehicle10relative to the remote devices48and/or the remote users49in possession of the remote devices48using at least one signal from one or more sensors40, such as cameras41, GNSS transceivers45(e.g., Global Positioning System (GPS) transceivers), data from the Internet, roadway databases and/or data from the other remote devices48, such as remote vehicles or remote mobile phones, via the communication transceivers37. The method100then proceeds to block106.

At block106, the controller34determines whether one or more social interests of the remote users48within the viewable area of the vehicle user11match one or more vehicle-user social interests of the vehicle user11using the social-networking data. The term “social interest” means an individual's awareness of belonging to a particular group. For example, a social interest may be that an individual belongs to a group that are fans of PINK FLOYD. In another example of a social interest, an individual may belong to a group that are HARVARD UNIVERSITY graduates. In yet another example of a social interest, an individual may belong to a group that are DETROIT TIGERS fans and are heading to a TIGERS baseball game. The term “vehicle-user social interest” means the social interest of the vehicle user11. The vehicle user11may input his or her social interests (i.e., the vehicle-user social interests) into the controller34through the user interface23before operating the host vehicle10. At block106, if the social interests of the remote users48within the viewable area of the vehicle user11do not match any of the vehicle-user social interests, then the method100proceeds to block108. At block108, the controller34performs no action, and the method100then returns to block104. If the social interests of one or more remote users48within the viewable area of the vehicle user11match one or more of the vehicle-user social interests, then the method100proceeds to block110.

At block110, the controller34determines the position (or location) of the host vehicle10relative to the remote users49having at least one social interest that matches the vehicle-user social interest. To do so, the controller34may use data received from GNSS transceiver45and/or roadway data, such as roadway databases. Stated differently, at block110, the controller34determines where the host vehicle10is located with respect to the remote users48and/or the remote devices48. Because the remote users49are in possession of remote devices48, the controller34may determine the location of the remote users49relative to the location of the host vehicle10using, for example, data received from the GNSS transceiver45, another sensor40, or another suitable source of information, such as the remote devices48. Then, the method100continues to block112.

At block112, the controller34determines the location of the eyes66and/or the head69of the user of the host vehicle10using at least one input from the user tracker43. As discussed above, the user tracker43may be a camera41configured to track the movements of the head69and/or the eyes66of the vehicle user. The controller34then uses the inputs from the user tracker43to continuously determine, in real time, the location of the eyes66and/or the head69of the user of the host vehicle10. The method100then proceeds to block116.

At block116, the controller34determines, in real time, the location, type, size, shape, and color of the virtual image50(FIG.2) to be displayed on the display29(e.g., the multi-focal plane augmented reality display) based on the location of the eyes66and/or the head69of the user of the host vehicle10, the social-networking data, and/or the location of the host vehicle10relative to the location of the remote users49. As a non-limiting example, the location of the virtual image50in the display29may change as the vehicle user11moves the head69and/or eyes66. Further, the size of the virtual images50may increase as the host vehicle10gets closer to the remote user49having at least one social interest that matches the vehicle-user social interests. Next, the method100proceeds to block118.

At block118, the controller34transmits a command signal to command the multi-focal plane augmented reality display29ato present at least one virtual image50at the previously determined locations. As discussed above, the virtual image50is indicative that the vehicle-user social interest that matches the social interest of the remote user49. Each virtual image50includes text, logo, slogan (e.g., DETROIT TIGERS logo, PINK FLOYD brand, and/or HARVARD logo), or other information indicative of the social interest of the particular remote user49that matches the vehicle-user social interest. The location of the virtual images50may change in real time based on location of the eyes66and/or the head69of the vehicle user. In response to receiving the command signal from the controller34, the multi-focal plane augmented reality display29ashows the virtual image50at the previously determined location with the previously determined size.

Also at block118, the controller34transmits a command signal to the information display29bto show the contact information76associated with each remote user49identified in the multi-focal plane augmented reality display29aas having the same social interest as the vehicle user11. To this end, the display29bshows remote-user information75and the contact information76of the particular remote user49next to the remote-user information75. The contact information76may be the page of the remote user49in a particular Internet-based social media platform, such as FACEBOOK, TWITTER, INSTAGRAM, WHATSAPP, and/or TIKTOK. The remote-user information75may include text identifying the remote user49having the same social interest as the vehicle user11. For example, the remote-user information75may include text, such as “PINK FLOYD fan”, “TIGER fan heading to the DODGER Stadium” and/or “HARVARD alumni '09”. Then, the method100continues to block120.

At block120, the controller34determines whether the remote user49with at least one social interest of the remote user that matches the vehicle-user social interest is still within the viewable area of the vehicle user11. To do so, the controller34determines: (1) whether the remote device48of the remote user49with at least one social interest of one or more remote users49that matches the vehicle-user social interest is within the predetermined viewable distance from the host vehicle10; (2) whether the remote device48of the remote user49with at least one social interest that matches the vehicle-user social interest is in the area in front of the host vehicle10; and (3) whether the remote device48and/or the remote user49with at least one social interest that matches the vehicle-user social interest is visible by the vehicle user11(FIG.6) through a windshield39of the host vehicle10. The controller34may use data from the user tracker43, the social-networking data, and/or data from the sensors40, such as the camera41to determine whether the remote user49with at least one social interest of the remote user that matches the vehicle-user social interest is still within the viewable area of the vehicle user11. If the remote user49with at least one social interest of the remote user49that matches the vehicle-user social interest is still within the viewable area of the vehicle user11, then the method100returns to block110. If the remote user49with at least one social interest of the remote users49that matches the vehicle-user social interest is not within the viewable area of the vehicle user11, then controller34commands the information display29band the multi-focal plane augmented reality display29ato stop presenting the information and virtual images50, respectively, of the remote users49that are no longer within the viewable area of the vehicle user11and the method100returns to block104. In other words, the controller34transmits an off signal to the multi-focal plane augmented reality display29ato discontinue showing the virtual image50on the multi-focal plane augmented reality display29a. The controller34also transmits an off signal to the information display29bto stop showing the contact information76of the remote users49that are no longer within the viewable area of the vehicle user11.

The drawings are in simplified form and are not to precise scale. For purposes of convenience and clarity only, directional terms such as top, bottom, left, right, up, over, above, below, beneath, rear, and front, may be used with respect to the drawings. These and similar directional terms are not to be construed to limit the scope of the disclosure in any manner.