Autonomous vehicle interaction system

A system for interacting with an autonomous vehicle includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a national stage application of the international application titled, “AUTONOMOUS VEHICLE INTERACTION SYSTEM,” filed on Aug. 9, 2019 and having application number PCT/US2019/046045. The subject matter of this related application is hereby incorporated herein by reference.

BACKGROUND

Field of the Embodiments

Embodiments of the present disclosure relate generally to autonomous vehicles and, more specifically, to an autonomous vehicle interaction system.

Description of the Related Art

Autonomous vehicles, which are capable of sensing their surroundings and moving with little or no human input, are being developed with the intention of making travel safer and easier. For example, fleets of self-driving cars may soon be employed as automated ride-hailing services.

Self-driving cars generally rely on a variety of sensors to identify appropriate navigation paths, potential obstacles, and relevant signage. For example, technologies commonly incorporated in self-driving cars include radar, LIDAR, sonar, global-positioning system (GPS) based navigation, odometry, and inertial metrology. Being thus equipped, driverless cars are well-suited to detect and avoid pedestrians and other persons in or proximate to roadways and other navigation paths.

Without a driver, however, once a self-driving car arrives at a requested or specifically targeted location, interaction between the self-driving car and outside passengers or pedestrians can be problematic. For example, unless a requesting rider is in possession of a device specifically designed to communicate with the arriving self-driving car, the self-driving car cannot distinguish the requesting rider from other persons in the vicinity.

In light of the above, more effective techniques for interacting with self-driving cars and other autonomous vehicles would be useful.

SUMMARY

The various embodiments set forth a system for interacting with an autonomous vehicle. The system includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted

At least one advantage of the disclosed embodiments is that a rider requesting service from an autonomous vehicle can readily discern a location at which to wait for the autonomous vehicle in order to interact with the vehicle. Another advantage is that, when positioned at the location, a rider or other user can communicate directly with the autonomous vehicle easily, intuitively, and without specialized communication equipment, for example by a gesture or verbal utterance. A further advantage is that a rider or other user can receive a visual and/or audible acknowledgment from the autonomous vehicle when the vehicle has detected the rider/user and/or has received a communication from the rider/user.

DETAILED DESCRIPTION

FIG.1is a schematic diagram illustrating an autonomous vehicle (AV) interaction system100, configured to implement one or more aspects of the present disclosure, according to various embodiments. AV interaction system100enables interactions between an autonomous vehicle101and a rider102or prospective passenger, without the need for rider102to be equipped with any sort of specialized communication device. Such interactions can include, without limitation, AV101detecting rider102, distinguishing rider102from other persons in the vicinity of rider102, determining that a ride request is being performed by rider102, receiving ride request information from rider102, and/or generating an acknowledgment indicator that is audible and/or visible to rider102. Further interactions enabled by AV interaction system100include, without limitation, rider102generating a ride request via a physical gesture and/or a verbal utterance and rider102receiving the acknowledgment indicator from AV101. The acknowledgment indicator can be generated by AV101when AV101determines a ride request has been generated by rider102and/or when AV101detects rider102.

In operation, AV interaction system100causes a hailing area150to be projected onto a suitable surface, and, when rider102is present within a boundary of hailing area150, causes a rider detection system to determine when rider102generates a ride request. According to various embodiments, hailing area150is a region that is explicitly designated for rider-to-AV and/or AV-to-rider interaction. Thus, when a person steps into the temporary “hot spot” indicated by hailing area150, a suitably configured AV can detect the person and focus optical and/or audio sensors onto the person, for example, in order to listen for voice commands and/or watch for hand or arm gestures. AV interaction system100includes, without limitation, a computing device120, a sensor or sensors130, and, in some embodiments, a visual projection system140.

AV101can be any computer-operated vehicle, such as an autonomous or self-driving car, truck, watercraft, vertical take-off and landing (VTOL) taxi, drone, and the like. In the embodiment illustrated inFIG.1, AV101is self-driving car, such as an autonomous taxi, that includes an AV control system103. AV101is capable of sensing the surrounding environment, both on and adjacent to the roadway, determining a suitable navigation path through the environment, and moving with little or no human input. As employed herein, a navigation path can include, without limitation, any street, roadway, driveway, parking lot automobile aisle, or other pathway suitable for travel by AV101. One embodiment of AV101is described below in conjunction withFIG.2.

FIG.2is a schematic diagram of AV101, according to one or more embodiments of the present disclosure. AV101includes, without limitation, AV sensors230, AV control system103, and computing device120. In some embodiments, AV101further includes visual projection system140and/or an acknowledgment output system210.

Acknowledgment output system210is configured to provide information to a specific rider102to acknowledge one or more specific conditions, such as detection of rider102within hailing area150and/or receipt of a ride request from rider102. Acknowledgment output system210can include, without limitation, one or more output devices to produce such an acknowledgment. For example, acknowledgment output system210can include a video screen or other display device for displaying a text-based and/or icon-based message to rider102, a color-coded light system for indicating a certain status of AV101, e.g., “occupied,” “available,” “on-route,” “stopping now,” and the like. Alternatively or additionally, acknowledgment output system210can include an audio output device, such as a directional speaker, that produces an audible acknowledgment to rider102. For example, a certain sound or audible word(s) can be directed to rider102when AV control system103detects rider102in hailing area150and a different sound or audible word(s) can be directed to rider102when AV control system103receives a ride request from rider102. Alternatively or additionally, acknowledgment output system210can include a haptic output device, such as an ultrasonic transducer, that produces a haptic acknowledgment to rider102. For example, the ultrasonic transducer can direct an ultrasonic signal onto a surface of rider102that is demodulated as a tactile vibration or pressure. Thus, in an embodiment, the ultrasonic transducer can cause rider102to experience a certain haptically-induced pressure or sensation, as an acknowledgement indicator, such as a haptic tap on the body. Alternatively, in some embodiments, the haptic output device can include an air puff generator configured to cause rider102to experience a haptic tap.

AV sensors230can include, without limitation, an array of different sensors configured to measure various properties associated with the environment surrounding AV101, including, but not limited to, a roadway or navigation path currently occupied by AV101, and/or areas adjacent to such a roadway or navigation path. AV sensors230can include, without limitation, optical sensors (visible light or infrared), acoustic sensors (such as ultrasound sensors, active sonar, and the like), RADAR sensors, LIDAR sensors, depth sensors, stereoscopic imaging sensors, topography mapping sensors, telematic sensors, receivers, and satellite-based navigation systems, and so forth. AV sensors230are configured to receive sensor data from a 360° panorama surrounding AV101and to transmit the sensor data to AV control system103for processing.

AV control system103is a computer-based system that is configured to manage the overall operation and navigation of AV101. AV control system103may include any technically feasible type of computer system, although in some embodiments, AV control system103can include a computing device similar to computing device120, which is described in greater detail below in conjunction withFIG.3. In operation, AV control system103is configured to receive sensor data from AV sensors230and to process that data to identify a current driving action and navigation path for AV101. In the context of this disclosure, a “driving action” may include turning, merging, driving through traffic, maintaining speed and lane position, caravanning with other vehicles, navigating to a specified location, and/or other actions associated with the operation of a vehicle. AV control system103can also be configured to process the sensor data from AV sensors230to identify potentially dangerous driving conditions associated with a driving action that is currently underway. Typically, such driving conditions include the presence, position, and velocity of other road users, such as vehicles, bicycles, scooters, Segways, or pedestrians, whether on the road or a sidewalk proximate the road. Alternatively, in some embodiments, detection of the presence of pedestrians, vehicles, and other potentially dangerous driving conditions can be performed by AV sensors230. Alternatively, in some embodiments, such functionality can be shared by or divided between AV sensors230and AV control system103.

Returning toFIG.1, sensors130enable interactions between AV101and rider102, and include, without limitation, one or more cameras or microphones that can generate image, video, and/or audio data associated with hailing area150and/or a person or persons disposed within hailing area150. Alternatively or additionally, in some embodiments, sensors130include proximity sensors directed toward hailing area150. In some embodiments, sensors130include directional sensors and/or sensors having variable orientation, such as a movable camera or a directional microphone. In such embodiments, as AV101approaches an area that includes one or more potential riders102, the orientation of sensors130can be changed, so that image, video, and/or audio data generated by sensors130is for a person or persons within hailing area150and includes little or no image, video, and/or audio data for a person or persons outside hailing area150.

In embodiments in which at least a portion of AV interaction system100is incorporated in AV101, sensors130are also included in AV101, for example, incorporated in AV101, coupled to an exterior surface of AV101, etc. Additionally, in such embodiments, sensors130may include or are coupled to AV sensors230. Thus, in such embodiments, image, video, and/or audio data generated by sensors130can include data already generated by AV sensors230.

In embodiments in which a portion of AV interaction system100is external to AV101, some of sensors130can be included in a fixed structure (not shown inFIG.1) that is external to AV101. For example, the fixed structure can be a designated pick-up area for autonomous vehicle riders that includes hailing area150. In such embodiments, a video camera and/or a microphone included in sensors103can be included in the fixed structure, thereby enabling a ride request from rider102being communicated to one or more AVs101that are not necessarily in the vicinity of hailing area150.

Visual projection system140is an optical projection system configured to direct light onto a suitable surface, such as a portion of sidewalk proximate AV101, and thus generate hailing area150. For example, in some embodiments, visual projection system140is configured to project an image or images, a geometrical pattern or shape, and/or a series of lines and/or curves onto a portion of sidewalk to generate hailing area150. In such embodiments, hailing area150is produced by the projection of light. Consequently, the geometrical patterns and/or shapes that form hailing area150can change location relative to AV101and/or be altered in appearance. For example, in such embodiments, some or all portions of hailing area150can flash, change color, or include animated portions as part of an acknowledgment indicator that is visible to rider102in certain circumstances. In some embodiments, AV control system103causes hailing area150to be generated proactively, for example to solicit acceptance of a ride from rider102. In such embodiments, such proactive rider acquisition can include the generation of hailing area150around a prospective rider150and the attempt to detect certain verbal utterances or physical gestures indicating acceptance or denial of the proffered ride.

Visual projection system140can include, without limitation a laser projection system, a liquid crystal display (LCD) projector, a three-dimensional (3D) capable projector, a video projector, or any other technically feasible system capable of projecting images onto a surface, such as a portion of sidewalk. In some embodiments, visual projection system140is configured to employ projection mapping to project images on such surfaces, and includes one or more depth sensors for sensing the area on which to project images. Further, in such embodiments, visual projection system140is configured to project images that compensate for detected irregularities of the projection surface. Thus, in such embodiments, corrected images are projected on the projection surface by visual projection system140that compensate for an irregular shape (curved, slanted, etc.) of the projection surface and/or for the presence of obstructions or other objects (benches, trash cans, rocks, etc.) on the projection surface. In this way, projection of, for example a horizontal square, is modified so that rider102is presented with a corrected image that approximates in appear the original image to be projected despite the irregular shape of or presence of obstructions on the surface.

Computing device120can be any computational platform configured to execute instructions for the operation of AV interaction system100, interaction sensor130, and visual projection system140, as described herein. One embodiment of computing device120is described below in conjunction withFIG.3.

FIG.3is a conceptual block diagram of computing device120, according to an embodiment of the present disclosure. Computing device120is configured to implement at least one aspect of the present disclosure described herein. Computing device120may be any type of device capable of executing application programs including, without limitation, instructions associated with a rider interaction application321, a sensor module322, a projection system module323, a gesture recognition module324, and/or a voice recognition module325. For example, and without limitation, computing device120may be an electronic tablet, a smartphone, a laptop computer, etc. Alternatively, computing device120may be implemented as a stand-alone chip, such as a microprocessor, or as part of a more comprehensive solution that is implemented as an application-specific integrated circuit (ASIC), a system-on-a-chip (SoC), and so forth. Generally, computing device120is configured to coordinate the overall operation of AV interaction system100. As shown, computing device120includes, without limitation, a processing unit310, input/output (I/O) devices320, and a memory device330.

Processing unit310may be implemented as a central processing unit (CPU), a graphics processing unit (GPU), an ASIC, a field programmable gate array (FPGA), any other type of processing unit, or a combination of different processing units. In general, processing unit310may be any technically feasible hardware unit capable of processing data and/or executing software applications to facilitate operation of AV interaction system100ofFIG.1, as described herein. Processing unit310may be physically embedded in computing device120, may be part of a cloud-based computing environment, and/or may be part of a computing device external to AV interaction system100, such as a mobile computing device or a wearable computing device. In some embodiments, processing unit310is an element of an in-vehicle infotainment system included in AV101, or is an element of AV control system103. Among other things, and without limitation, processing unit310may be configured to execute instructions associated with rider interaction application321, sensor module322, projection system module323, gesture recognition module324, and/or voice recognition module325.

Rider interaction application321is configured to manage the operation of AV interaction system100. In some embodiments, rider interaction application321receives image and/or audio data from sensors130, interprets such data, and determines whether rider102has performed a ride request. For example, in such embodiments, rider interaction application321may receive image data from sensors130and determine, via gesture recognition module324, whether rider102has executed a gesture indicating a request for a ride from AV101. Alternatively or additionally, in such embodiments, rider interaction application321may receive sound data from sensors130and determine, via voice recognition module325, whether rider102has pronounced a verbal utterance indicating a request for a ride from AV101. In such embodiments, one or more directional microphones included in sensors130can provide selective audio data that substantially reduces and/or eliminates verbal utterances pronounced by persons outside of hailing area150.

In addition, in some embodiments, rider interaction application321receives image data from sensors130, interprets such data, and determines whether rider102is located within hailing area150. Thus, in such embodiments, rider interaction application321distinguishes rider102from other persons in the vicinity of rider102, based on rider102being disposed within hailing area150. Alternatively or additionally, in some embodiments, rider interaction application321generates an acknowledgment indicator that is audible and/or visible to rider102. In such embodiments, the acknowledgment indicator can be a visual, audible, and/or haptic acknowledgment generated by acknowledgment output system210. Alternatively or additionally, in embodiments in which hailing area150is a projected hailing area generated by visual projection system140, the acknowledgment indicator can include a change in the appearance of hailing area150, such as a change in color, the flashing or animation of certain portions of hailing area150, etc. In the case of a haptic acknowledgement, an ultrasonic transducer included in acknowledgment output system210cause rider102to experience a haptic tap or other tactile acknowledgement.

In some embodiments, rider interaction application321includes an application programming interface (API) configured to facilitate communication between computing device120, AV control system103, and/or a ride-hailing administration entity that is external to AV101and AV interaction system100. For example, the API can enable ride request information received by rider interaction application321to be communicated to the ride-hailing administration entity for further distribution to other AVs in the vicinity of hailing area150.

Sensor module322is configured to control operation of sensors130, including receipt of image and/or audio data. In embodiments in which sensors130include one or more directional sensors, such as a movable camera or a directional microphone, sensor module322can also be configured to control the current orientation of such directional sensors. Projection system module323is configured to control operation of visual projection system140, including orientation thereof to a suitable surface for hailing area150. Gesture recognition module324is configured to receive image data from sensors130and determine, based on such image data, whether rider102has performed a physical hand, arm, head, or other gesture that corresponds to a ride request. Examples of recognizable gestures include, without limitation, a hailing gesture, indicating that rider102wants a ride, and a wave-off gesture, indicating the rider102does not want a ride. The wave-off gesture may be recognized in situations in which AV control system103has implicitly or explicitly solicited rider102, for example when AV control system103causes hailing area150to be generated proactively. The wave-off gesture may also be recognized in situations in a person inadvertently stands in hailing area150despite not wanting a ride. Gesture recognition module324is further configured to inform rider interaction application321when such a gesture has been detected. Voice recognition module325is configured to receive audio data from sensors130and determine, based on such image audio data, whether rider102has pronounced a verbal utterance that corresponds to a ride request, such as “I would like a ride to downtown,” or “I need a ride.” Voice recognition module325is further configured to inform rider interaction application321when such a verbal utterance has been detected and, in some embodiments, additional information included in the ride request, such as destination information.

Memory330may include a random access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof, and may include a single memory module or a collection of memory modules. As shown, in some embodiments, some or all of rider interaction application321, sensor module322, projection system module323, gesture recognition module324, and/or voice recognition module325may reside in memory330during operation.

I/O devices320includes at least one device capable of both receiving input, such as a keyboard, a mouse, a touch-sensitive screen, a microphone and so forth, as well as devices capable of providing output, such as a display screen, loudspeakers (including a loudspeaker associated with a headphone-based system), and the like. The display screen may be incorporated in AV interaction system100or may be external to AV interaction system100, such as a computer monitor, a video display screen, a display apparatus incorporated into a separate hand held device, or any other technically feasible display screen. In some embodiments, I/O devices320include one or more of sensors130and/or one or more of AV sensors230.

In the embodiment illustrated inFIG.3, computing device120is shown as a single integrated device. In other embodiments, some or all of the functionality of computing device120described herein can be implemented as multiple computing devices, such as a computing device integrated into sensors130, a computing device integrated into AV control system103, and/or a computing device integrated into visual projection system140.

In some embodiments, hailing area150is projected onto a fixed location on a projection surface while AV101is in motion. In such embodiments, hailing area150can remain stationary as AV101moves relative thereto. One such embodiment is illustrated inFIGS.4A-4C.

FIGS.4A-4Cschematically illustrate a hailing area450generated by an AV-mounted projection system, according to an embodiment of the present disclosure. For example, in some embodiments, hailing area450can be projected onto a suitable surface by a roof-mounted projection system and/or by a projection system incorporated into a suitably configured headlight. An example of each such projection system140is illustrated inFIG.2. As shown, when hailing area450is projected by an AV-mounted projection system, hailing area450can be generated so that hailing area450remains stationary, even while AV101is in motion. That is, as AV101drives along a roadway402or other navigation path past a particular location401that is disposed adjacent to roadway402, hailing area450is projected from AV101so that location401is disposed within hailing area450. In such embodiments, the projection mapping capability of visual projection system140enables the projection of stationary hailing area450.

For example, as AV101slowly drives past an area in which multiple potential riders are standing, hailing area450is projected to remain substantially stationary relative to the potential riders, and therefore is in motion relative to AV101. Because hailing area450is stationary relative to location401, any potential rider near location401can immediately see hailing area450. Further, when the potential rider wants to hail AV101, the potential rider can easily step into hailing area450, since hailing area450is stationary while AV101drives past. Typically, AV101is traveling at a relatively low speed when in the vicinity of location401and, as a result, the potential rider has ample time to step into hailing area450before AV101has moved on toward a different location for rider pickup.

In some embodiments, location401is selected by AV interaction system100based on one or more factors. For example, in some embodiments, AV interaction system100selects location401in response to detecting one or more potential riders102proximate location401. Thus, in one such embodiment, AV interaction system100selects location401in response to detecting a pedestrian proximate roadway402and proactively attempts to acquire a rider102. In another such embodiment, location401is a specific location or category of location, such as a bus stop or airport taxi zone, and AV interaction system100selects location401in response to rider102being present in such a specific location. Alternatively or additionally, in some embodiments, AV interaction system100selects location401in response to detecting that location401corresponds to portion of roadway402in which temporary parking, stopping, and/or loading/unloading is permitted. Alternatively or additionally, in some embodiments, AV interaction system100selects location401in response to receiving instructions specifying that AV101proceed to location401, for example from a ride-hailing administration entity that is external AV101and AV interaction system100. Alternatively or additionally, in some embodiments, AV interaction system100selects location401in response to receiving a request from a particular rider to meet at or near location401.

In some embodiments, portions of an AV interaction system are incorporated into a fixed structure that is external to AV101. For example, in some embodiments, one or more sensors130ofFIG.1and/or projection system140ofFIG.1are incorporated into a fixed structure, such as an AV ride-hailing location. One such embodiment is described below in conjunction withFIG.5.

FIG.5schematically illustrates an AV interaction system500that is distributed between one or more AVs501and a fixed structure510, according to an embodiment of the present disclosure. AV501is substantially similar to AV101ofFIG.1, except that one or more components of AV interaction system100are not included in AV501, but are instead incorporated into fixed structure510. In the embodiment illustrated inFIG.5, AV501includes sensors530, acknowledgment output system210, AV control system103and computing device120. Fixed structure510includes sensors531, a visual projection system540, and a computing device520that includes or is communicatively coupled to a communication module525. Fixed structure510can be located proximate any suitable roadway or navigation path, such as near the curb side at an airport or train station.

Sensors530of AV501are configured to enable interactions between AV501and a rider502located within hailing area550, such as the detection of gestures by rider502. As such, sensors530can include some or all of the sensors employed by AV501for navigation and operation, as well as some or all of the sensors described above for sensors130ofFIG.1. For example, in an embodiment, AV501employs optical sensors that are already configured for use with AV501for navigation to generate image data, and additionally includes one or more microphones for receiving audible input from rider502as audio data.

In contrast to sensors530, sensors531are configured to enable interactions between rider502and the portion of AV interaction system500included in fixed structure510. Sensors531can include one or more cameras or microphones that can generate image, video, and/or audio data associated with hailing area550and/or a person or persons disposed within hailing area550, i.e. rider502. Thus, sensors531enable a person to interact with AV interaction system500by stepping within the boundaries of hailing area550and performing a specified physical gesture (e.g., a hand or arm signal) or pronouncing a specified verbal utterance (e.g., “I need a ride”). In this way, rider502can request a ride from AV501and/or from a fleet of AVs, even though no AV is within line of sight of fixed structure510or located in the vicinity of fixed structure510.

Computing device520is configured to perform certain functions of AV interaction system500. Such functions can include, without limitation, detection of a person within hailing area550, determination that a ride request is performed by the person within hailing area550, generation of a ride request signal that causes an autonomous vehicle to approach hailing area550, and/or direction of an indicator output to the person within hailing area550, such as an audible and/or visible acknowledgment indicator. In some embodiments, computing device520is configured to determine a person has performed a ride request based on detecting a gesture from image data generated by sensors531and/or on detecting a certain verbal utterance from audio data generated by sensors531. The audible and/or visible acknowledgment indicator for the person within hailing area550can include, without limitation, an alteration in appearance of hailing area550or the direction of a certain sound or audible word(s) to rider502. For example, some or all portions of hailing area550can flash, change color, or include animated portions as part of an acknowledgment indicator that is visible to rider502.

Communication module525is configured to enable communications between computing device520and computing device120, via any technically feasible wireless communication system. For example, in some embodiments, communication module525is configured to enable direct communication between computing device520and computing device120via radio waves. Alternatively or additionally, in some embodiments, communication module525is configured to enable wireless communication between computing device520and computing device120via cellular communications. Alternatively or additionally, in some embodiments, communication module525is configured to enable communication between computing device520and computing device120via a ride-hailing administration entity560that is external to AV interaction system500.

In the embodiment illustrated inFIG.5, computing device120is also configured to perform certain functions of AV interaction system500, but generally not all of the functions described above in conjunction withFIG.1. For example, in the embodiment illustrated inFIG.5, computing device120is not configured to cause a projection system to generate hailing area550. Additionally or alternatively, computing device120can be configured to receive a ride request signal indicating hailing area550as a pickup location. In such embodiments, the ride request signal can be received directly from communication module525via wireless communication or from ride-hailing administration entity560, such as a communication system that is associated with a fleet or multiple fleets of AVs. Furthermore, in some embodiments, computing device120is configured to detect rider502when approaching hailing area550, for example via gesture detection and/or face recognition. In such embodiments, computing device120can be configured to provide information to a specific rider502via output system210as an acknowledgment and/or to inform the specific rider502that AV501is designated for that specific rider502.

In some embodiments, hailing area550is projected temporarily, for example when a rider is within a certain distance of fixed structure510. Alternatively, hailing area550is indicated via a permanent indicator, such as markings delineating the boundaries of hailing area550, appropriate signage, and/or a continuous projection from projection system540. In some embodiments, computing device520is further configured to generate a first acknowledgment indicator when rider502is initially detected within hailing area550and a second acknowledgment indicator when a ride request by rider502is detected. In such embodiments, the first and second acknowledgment indicators can be audible and/or visible indicators as described above in conjunction withFIG.1.

FIG.6is a flowchart of method steps for facilitating interaction with an autonomous vehicle, according to various embodiments of the present disclosure. Although the method steps are described with respect to the systems ofFIGS.1-5, persons skilled in the art will understand that any system configured to perform the method steps, in any order, falls within the scope of the various embodiments.

As shown, a method600begins at step601, in which an AV interaction system, e.g., AV interaction system100or AV interaction system500, generates a hailing area. In embodiments in which AV101includes visual projection system140, visual projection system140directs light onto a suitable surface, such as a portion of sidewalk proximate AV101, to generate hailing area150. In such embodiments, the AV interaction system may generate hailing area150under specific circumstances, such as when AV101arrives at a specified location. In embodiments in which an AV interaction system includes fixed structure510with visual projection system540, the AV interaction system may generate hailing area550continuously or when a person is detected to be in the vicinity of fixed structure510.

In step602, the AV interaction system detects a person within the hailing area. In some embodiments, a computing device120included in AV101detects rider102within hailing area150, which is projected from AV101. In such embodiments, computing device120makes such a determination based on image and/or other sensor data from sensors130. In other embodiments, computing device520detects rider502within hailing area550, which is disposed proximate fixed structure510. In such embodiments, computing device520makes such a determination based on image and/or other sensor data from sensors530.

In step603, the AV interaction system generates a first acknowledgment indicator, such as an acknowledgment indicator that is visible and/or audible to rider102in hailing area150or rider502in hailing area550. The AV interaction system generates the first acknowledgment indicator in response to detecting the person within the hailing area. Thus, the person is made aware that the AV interaction system is monitoring the hailing area to determine whether the person has performed a ride request.

In step604, the AV interaction system determines that a ride request has been performed by the person in the hailing area. All persons outside the hailing area are ignored by the AV interaction system. Therefore, determining that a ride request has been performed is greatly simplified computationally, since the gestures and/or verbal utterances of only persons within the hailing area are analyzed. In embodiments in which AV interaction system is incorporated into AV101, computing device120determines that the ride request has been performed based on image and/or audio data from sensors130. Alternatively, in embodiments in which AV interaction system500is distributed between one or more AVs501and a fixed structure510, computing device520determines that the ride request has been performed based on image and/or audio data from sensors530.

In some embodiments, the AV interaction system receives additional ride request information, such as a rider request for a specific model or style of AV, an AV that can arrive in the shortest time, an AV that can provide a ride for the lowest fair, an AV belonging to a specific fleet of AVs, etc. For example, in some embodiments, the AV interaction system receives the additional ride request information via audible commands from the rider, from a mobile computing device associated with the rider, from a matrix barcode or other machine-readable optical label associated with the rider, etc.

In step605, the AV interaction system generates a ride request signal in response to determining that the ride request has been performed in step604. In embodiments in which AV interaction system is incorporated into AV101, computing device120generates and transmits the ride request signal to AV control system103, and AV101responds accordingly. For example, AV101safely approaches rider102. Alternatively, in embodiments in which AV interaction system500is distributed between one or more AVs501and a fixed structure510, computing device520transmits the ride request to the one or more AVs501directly or via ride-hailing administration entity560. In either case, at least one AV501responds accordingly and navigates to fixed structure510.

In step606, the AV interaction system directs an output to the person in the hailing area. Thus, the potential rider can readily determine that the ride request has been received by the AV interaction system. For example, the AV interaction system causes a second acknowledgment indicator to be generated, such as a visible or audible acknowledgment that the ride request from the person in the hailing area has been received. In embodiments in which AV interaction system is incorporated into AV101, computing device120causes the second acknowledgment indicator to be generated by acknowledgment output system210. Alternatively, in embodiments in which AV interaction system500is distributed between one or more AVs501and a fixed structure510, computing device520causes the second acknowledgment indicator to be generated by visual projection system540, a loudspeaker associated with fixed structure510, etc.

In sum, various embodiments set forth systems and techniques for interacting with an autonomous vehicle. In the embodiments, a specified hailing area is monitored for the presence of a person by suitable sensors and, when a person is detected within the hailing area, the sensors enable detection of the person performing a ride request. In addition, a visible and/or audible acknowledgment is directed to the person within the hailing area when the person is first detected within the hailing area and/or when the ride request is detected. The hailing area can be a predesignated area or generated by a projection system included in an autonomous vehicle.

At least one technological improvement of the disclosed embodiments is that a rider requesting service from an autonomous vehicle can readily discern at what location to wait for the autonomous vehicle in order to interact with the vehicle. Another advantage is that, when disposed at the location, the rider requesting the service can communicate directly with the autonomous vehicle easily, intuitively, and without specialized communication equipment, for example by a gesture or verbal utterance. A further advantage is that the rider can receive visual and/or audible acknowledgment from the autonomous vehicle when the vehicle has detected the rider and/or has received a communication from the rider.

1. In some embodiments, a system for interacting with an autonomous vehicle includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted.

2. The system of clause 1, wherein the processor is further configured to execute instructions to cause the projection system to generate the projected hailing area on a surface proximate the autonomous vehicle.

3. The system of clauses 1 or 2, wherein the processor is further configured to execute instructions to cause the projection system to generate the projected hailing area on the surface proximate the autonomous vehicle in response to detecting that the autonomous vehicle is located in proximity to a specified location.

4. The system of any of clauses 1-3, wherein the processor is further configured to execute instructions to cause the projection system to generate the projected hailing area on a stationary portion of the surface while the autonomous vehicle is moving relative to the person.

5. The system of any of clauses 1-4, wherein the processor is further configured to execute instructions to, in response to receiving instructions to pick up a particular rider at the specified location, cause the autonomous vehicle to drive to the specified location.

6. The system of any of clauses 1-5, wherein the processor is further configured to execute instructions to determine that a ride request is performed by the person within the projected hailing area.

7. The system of any of clauses 1-6, wherein the ride request comprises a physical gesture performed by the person within the projected hailing area.

8. The system of any of clauses 1-7, wherein the processor is configured to execute the instructions to determine the ride request based on image data from the sensor.

9. The system of any of clauses 1-8, wherein the ride request comprises a verbal utterance by the person within the projected hailing area.

10. The system of any of clauses 1-9, wherein the processor is configured to execute the instructions to determine the ride request based on sound data from at least one microphone included in the sensor.

11. In some embodiments, a system for facilitating interactions with an autonomous vehicle includes: a sensor configured to generate sensor data corresponding to a hailing area; and a processor configured to execute instructions to: receive sensor input from the sensor; based on the sensor input, detect the person within the hailing area; and in response to detecting a person within the hailing area, generate a ride request signal that causes an autonomous vehicle to approach the hailing area; and cause an acknowledgment indicator to be outputted.

12. The system of clause 11, wherein the processor and the sensor are included in a fixed structure external to the autonomous vehicle.

13. The system of clauses 11 or 12, wherein detecting the person within the hailing area comprises determining that a ride request is performed by the person within the projected hailing area.

14. The system of any of clauses 1-13, wherein the ride request comprises one of a physical gesture performed by the person within the projected hailing area and a verbal utterance pronounced by the person within the projected hailing area.

15. The system of any of clauses 1-14, wherein generating the rider request signal comprises determining that the person within the hailing area has requested that the autonomous vehicle provide a ride to the person.

16. A method for facilitating interaction with an autonomous vehicle, the method comprising: projecting a hailing area on a surface proximate the autonomous vehicle detecting, via one or more sensors, a person disposed within the hailing area; determining that a ride request has been performed by the person; and generating a ride request signal that causes an autonomous vehicle to approach the hailing area and output an acknowledgement indicator to the person.

17. The method of clause 16, wherein generating the ride request signal comprises one of transmitting the ride request signal to the autonomous vehicle and transmitting the ride request signal to an administration entity that is external to the autonomous vehicle.

18. The method of clauses 16 or 17, further comprising, in response to detecting the person within the hailing area, adjusting the sensor to focus on the hailing area.

19. The system of any of clauses 16-18, wherein outputting the acknowledgment indicator to the person comprises at least one of generating an acknowledgment indicator that is visible to the person within the hailing area, generating an acknowledgment indicator that is audible to the person within the hailing area, and generating an acknowledgment indicator that can be felt by the person within the hailing area.

20. The system of any of clauses 16-19, wherein generating the acknowledgment indicator that is visible to the person within the hailing area comprises modifying a visual feature of the hailing area.

Aspects of the present embodiments may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “module” or “system.” In addition, any hardware and/or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in at least one computer readable medium having computer readable program code embodied thereon.