Patent Description:
As transportation moves towards autonomous (i.e., driverless) vehicles, the manufactures and designers of these autonomous vehicles must address issues that are not a concern in traditional vehicles. Unfortunately, autonomous vehicles are still a rarity on the streets and people are still unsure of the manner in which they operate.

For example, when a person is waiting to cross a street at a crosswalk and a vehicle is approaching the same crosswalk, the person would normally wait until there is a signal that the driver of the vehicle sees them. For example, the person wait until the driver of the other vehicle stares at them. or waves them on. or flashes their headlights at them.

However, an autonomous vehicle does not have a driver. Accordingly, there is no one in the vehicle that can provide the signal to the person waiting the cross the street at the crosswalk.

<CIT> discloses a method for autonomous vehicle-to-human communications. Upon detecting a human traffic participant being proximal to a traffic yield condition of a vehicle planned route, generating a message for broadcast to the human traffic participant and sensing whether the human traffic participant acknowledges a receipt of the message. When sensing that the human traffic participant acknowledges receipt of the message, generating a vehicle acknowledgment message for broadcast to the pedestrian.

<CIT> provides a method, and an autonomous moving body which allow a recognition target to be notified with certainty that the recognition target is recognized by an autonomous moving body. A recognition result presenting apparatus detects a recognition target e.g. a person present within a predetermined range from an automatic driving vehicle and presents, to the detected recognition target, the result of recognition indicating that the automatic driving vehicle recognizes the recognition target.

<CIT> discloses methods for implementing an active safety system in an autonomous vehicle. An autonomous vehicle may be travelling through an environment external to the autonomous vehicle along a trajectory. The environment may include one or more objects that may potentially collide with the autonomous vehicle, such as static and/or dynamic objects, or objects that pose some other danger to passengers riding in the autonomous vehicle and/or to the autonomous vehicle. An object e.g., an automobile is depicted as having a trajectory, that if not altered, may result in a potential collision with the autonomous vehicle. The autonomous vehicle may use a sensor system to sense the environment to detect the object and may take action to mitigate or prevent the potential collision of the object with the autonomous vehicle.

<CIT> relates generally to notifying a pedestrian of the intent of a self-driving vehicle. The vehicle may include sensors which detect an object such as a pedestrian attempting or about to cross the roadway in front of the vehicle. The vehicle's computer may then determine the correct way to respond to the pedestrian. The computer may determine that the vehicle should stop or slow down, yield, or stop if it is safe to do so. The vehicle may then provide a notification to the pedestrian of what the vehicle is going to or is currently doing. For example, the vehicle may include a physical signaling device, an electronic sign or lights, a speaker for providing audible notifications, etc..

<CIT> relates to a communication method of a motor vehicle with a traffic participant.

The problem to be solved is to provide a method with improved indication for pedestrians.

The problem is solved in one implementation, wherein a computer-implement method is executed on a computing device and includes: monitoring one or more machine vision sensors to obtain perception information concerning one or more pedestrians proximate an autonomous vehicle; identifying one or more humanoid shapes within the perception information, thus defining one or more detected humanoid shapes; generating proximate object display information that locates the one or more detected humanoid shapes with respect to the autonomous vehicle; and rendering the proximate object display information on a visual display system, thus confirming the perception of the one or more pedestrians by the autonomous vehicle, the rendering being made according to the rendering features of claim <NUM>.

One or more of the following features are included. Identifying one or more humanoid shapes within the perception information includes: comparing one or more defined humanoid shapes to one or more unidentified objects within the perception informationto identify one or more humanoid shapes within the perception information. The proximate object display information includes dynamic proximate object display information that changes as the location of the one or more detected humanoid shapes changes with respect to the autonomous vehicle. Rendering the proximate object display information on a visual display system includes: rendering the dynamic proximate object display information on the visual display system, thus dynamically confirming the perception of the one or more pedestrians by the autonomous vehicle. The one of more machine vision sensors may include aLIDAR system. The visual display system is configured to be mounted on a roof of the autonomous vehicle. The visual display system can be a cylindrical visual display system. The cylindrical visual display system includes: an illuminated portion; and a non-illuminated portion positioned between the illuminated portion and the roof of the autonomousvehicle. The visual display system is a <NUM> degree visual display system. The visual display system is integrated into the autonomous vehicle.

In another implementation, a computer program product resides on a computer readable medium and has a plurality of instructions stored on it. When executed by a processor, the instructions cause the processor to perform the steps of the above method.

In another implementation, a computing system includes a processor and memory is configured to perform the steps of the method.

Other features and advantages will become apparent fromthe description, the drawings, and the claims.

Referring to <FIG>, there is shown autonomous vehicle <NUM>. As is known in the art, an autonomous vehicle (e.g. autonomous vehicle <NUM>) is a vehicle that is capable of sensing its environment and moving with little or no human input. Autonomous vehicles (e.g. autonomous vehicle <NUM>) may combine a variety of sensor systems to perceive their surroundings, examples of which may include but are not limited to radar, computer vision, LIDAR, GPS, odometry, temperature and inertia, wherein such sensor systems may be configured to interpret lanes and markings on a roadway, street signs, stoplights, pedestrians, other vehicles, roadside objects, hazards, etc..

Autonomous vehicle <NUM> includes a plurality of sensors (e.g. sensors <NUM>), a plurality of electronic control units (e.g. ECUs <NUM>) and a plurality of actuators (e.g. actuators <NUM>). Accordingly, sensors <NUM> within autonomous vehicle <NUM> may monitor the environment in which autonomous vehicle <NUM> is operating, wherein sensors <NUM> may provide sensor data <NUM> to ECUs <NUM>. ECUs <NUM> may process sensor data <NUM> to determine the manner in which autonomous vehicle <NUM> should move. ECUs <NUM> may then provide control data <NUM> to actuators <NUM> so that autonomous vehicle <NUM> may move in the manner decided by ECUs <NUM>. For example, a machinevision sensor included within sensors <NUM> may "read" a speed limit sign stating that the speed limit on the road on which autonomous vehicle <NUM> is traveling is now <NUM> miles an hour. This machine vision sensor included within sensors <NUM> may provide sensor data <NUM> to ECUs <NUM> indicating that the speed on the road on which autonomous vehicle <NUM> is traveling is now <NUM> mph. Upon receiving sensor data <NUM>, ECUs <NUM> may process sensor data <NUM> and may determine that autonomous vehicle <NUM> (which is currently traveling at <NUM> mph) is traveling too fast and needs to slow down. Accordingly, ECUs <NUM> may provide control data <NUM> to actuators <NUM>, wherein control data <NUM> may e.g. apply the brakes of autonomous vehicle <NUM> or eliminate any actuation signal currently being applied to the accelerator (thus allowing autonomous vehicle <NUM> to coast until the speed of autonomous vehicle <NUM> is reduced to <NUM> mph).

As would be imagined, since autonomous vehicle <NUM> is being controlled by the various electronic systems included therein (e.g. sensors <NUM>, ECUs <NUM> and actuators <NUM>), the potential failure of one or more of these systems should be considered when designing autonomous vehicle <NUM> and appropriate contingency plans are employed.

For example and referring also to <FIG>, the various ECUs (e.g., ECUs <NUM>) that are included within autonomous vehicle <NUM> are compartmentalized so that the responsibilities of the various ECUs (e.g., ECUs <NUM>) may be logically grouped. For example, ECUs <NUM> includes autonomy control unit <NUM> that receives sensor data <NUM> from sensors <NUM>.

Autonomy control unit <NUM> is configured to perform various functions. For example, autonomy control unit <NUM> receives and processes exteroceptive sensor data (e.g., sensor data <NUM>), mestimates the position of autonomous vehicle <NUM> within its operating environment, calculates a representation of the surroundings of autonomous vehicle <NUM>, computes safe trajectories for autonomous vehicle <NUM>, and commands the other ECUs (in particular, a vehicle control unit) to cause autonomous vehicle <NUM> to execute a desired maneuver. Autonomy control unit <NUM> includes substantial compute power, persistent storage, and memory.

Accordingly, autonomy control unit <NUM> processes sensor data <NUM> to determine the manner in which autonomous vehicle <NUM> should be operating. Autonomy control unit <NUM> then provides vehicle control data <NUM> to vehicle control unit <NUM>, wherein vehicle control unit <NUM> then processes vehicle control data <NUM> to determine the manner in which the individual control systems (e.g. powertrain system <NUM>, braking system <NUM> and steering system <NUM>) should respond in order to achieve the trajectory defined by autonomous control unit <NUM> within vehicle control data <NUM>.

Vehicle control unit <NUM> is configured to control other ECUs included within autonomous vehicle <NUM>. For example, vehicle control unit <NUM> controls the steering, powertrain, and brake controller units. For example, vehicle control unit <NUM> provides: powertrain control signal <NUM> to powertrain control unit <NUM>; braking control signal <NUM> to brakingcontrol unit <NUM>; and steering control signal <NUM> to steering control unit <NUM>.

Powertrain control unit <NUM> processes powertrain control signal <NUM> so that the appropriate control data (commonly represented by control data <NUM>) are provided to powertrain system <NUM>. Additionally, braking control unit <NUM> processes braking control signal66 so that the appropriate control data (commonly represented by control data <NUM>) are provided to braking system <NUM>. Further, steering control unit <NUM> processes steering control signal <NUM> so that the appropriate control data (commonly represented by control data <NUM>) are provided to steering system <NUM>.

Powertrain control unit <NUM> is configured to control the transmission (not shown) and engine / traction motor (not shown) within autonomous vehicle <NUM>; while brake control unit <NUM> is configured to control the mechanical / regenerative braking system (notshown) within autonomous vehicle <NUM>; and steering control unit <NUM> is configured to control the steering column/ steering rack (not shown) within autonomous vehicle <NUM>.

Autonomy control unit <NUM> is a highly complex computing system that provides extensive processing capabilities (e.g., a workstation-class computing system with multi-core processors, discrete co-processing units, gigabytes of memory, and persistent storage). In contrast, vehicle control unit <NUM> is a much simpler device that provides processing power equivalent to the other ECUs included within autonomous vehicle <NUM> (e.g., acomputing system having a modest microprocessor (with a CPU frequency of less than <NUM> megahertz), less than <NUM> megabyte of system memory, and no persistent storage). Due to these simpler designs, vehicle control unit <NUM> has greater reliability and durability than autonomy control unit <NUM>.

To further enhance redundancy and reliability, one or more of the ECUs (ECUs <NUM>) included within autonomous vehicle <NUM> is configured in a redundant fashion. For example and referring also to <FIG>, there is shown one implementation of ECUs <NUM> wherein a plurality of vehicle control units are utilized. For example, this particular implementation is shown to include two vehicle control units, namely a first vehicle control unit (e.g., vehicle control unit <NUM>) and a second vehicle control unit (e.g., vehicle control unit <NUM>).

In this particular configuration, the two vehicle control units (e.g. vehicle control units <NUM>, <NUM>) are configured in various ways. For example, the two vehicle control units (e.g. vehicle control units <NUM>, <NUM>) are configured in an active - passive configuration, wherein e.g. vehicle control unit <NUM> performs the active role of processing vehicle control data <NUM> while vehicle control unit <NUM> assumes a passive role and is essentially in standby mode. In the event of a failure of vehicle control unit <NUM>, vehicle control unit <NUM> transitions from a passive role to an active role and assumes the role of processing vehicle control data <NUM>. Alternatively, the two vehicle control units (e.g. vehicle control units <NUM>, <NUM>) are configured in an active - active configuration, wherein e.g. both vehicle control unit <NUM> and vehicle control unit <NUM> perform the active role of processing vehicle control data <NUM> (e.g. divvying up the workload), wherein in the event of a failure of either vehicle control unit <NUM> orvehicle control unit <NUM>, the surviving vehicle control unit may process all of vehicle control data <NUM>.

While <FIG> illustrates one example of the manner in which the various ECUs (e.g. ECUs <NUM>) included within autonomous vehicle <NUM> are configured in a redundant fashion, this is for illustrative purposes only and is not intended to be a limitation of this invention, as other configurations are possible and are considered to be within the scope of this invention. For example, autonomous control unit <NUM> is configured in a redundant fashion, wherein a second autonomous control unit (not shown) is included within autonomous vehicle <NUM> and is configured in an active - passive or active - active fashion. Further, it is foreseeable that one or more of the sensors (e.g., sensors <NUM>) and/or one or more of the actuators (e.g. actuators <NUM>) may be configured in a redundant fashion. Accordingly, it is understood that the level of redundancy achievable with respect to autonomous vehicle <NUM> mayonly be limited by the design criteria and budget constraints of autonomous vehicle <NUM>.

Referring also to <FIG>, the various ECUs of autonomous vehicle <NUM> are grouped/ arranged/ configured to effectuate various functionalities.

For example, one or more of ECUs <NUM> are configured to effectuate I form perception subsystem <NUM>. wherein perception subsystem <NUM> is configured to process data from onboard sensors (e.g., sensor data <NUM>) to calculate concise representations of objects of interest near autonomous vehicle <NUM> (examples of which includes but are not limited to other vehicles, pedestrians, traffic signals, traffic signs, road markers, hazards, etc.) and to identify environmental features that assist in determining the location of autonomous vehicle <NUM>. Further, one or more of ECUs <NUM> are configured to effectuate / form state estimation subsystem <NUM>, wherein state estimation subsystem <NUM> is configured to process data from onboard sensors (e.g., sensor data <NUM>) to estimate the position, orientation, and velocity of autonomous vehicle <NUM> within its operating environment. Additionally, one or more of ECUs <NUM> are configured to effectuate I form planning subsystem <NUM>, wherein planning subsystem <NUM> are configured to calculate a desired vehicle trajectory (using perception output <NUM> and state estimation output <NUM>). Further still, one or more of ECUs <NUM> are configured to effectuate I form trajectory control subsystem <NUM>, wherein trajectory control subsystem <NUM> uses planning output <NUM> and state estimation output <NUM> (in conjunction with feedback and/or feedforward control techniques) to calculate actuator commands (e.g., control data <NUM>) that cause autonomous vehicle <NUM> to execute its intended trajectory within it operating environment.

For redundancy purposes, the above-described subsystems are distributed across various devices (e.g., autonomy control unit <NUM> and vehicle control units <NUM>, <NUM>). Additionally / alternatively and due to the increased computational requirements, perception subsystem <NUM> and planning subsystem <NUM> are located almost entirely within autonomy control unit <NUM>, which (as discussed above) has much more computational horsepower than vehicle control units <NUM>, <NUM>. Conversely and due to their lower computational requirements, state estimation subsystem <NUM> and trajectory control subsystem <NUM> are located entirely on vehicle control units <NUM>, <NUM> if vehicle control units <NUM>, <NUM> have the requisite computational capacity; and/or located partially on vehicle control units <NUM>, <NUM> and partially on autonomy control unit <NUM>. However, the location of state estimation subsystem <NUM> and trajectory controlsubsystem <NUM> are of critical importance in the design of any contingency planning architecture, as the location of these subsystems determine how contingency plans arecalculated, transmitted, and/or executed.

Referring also to <FIG>, <FIG>, <FIG>, there is shown an exterior view of autonomousvehicle <NUM>, wherein autonomous vehicle <NUM> includes status indication system <NUM> for conveying status information concerning a moveable vehicle (e.g., autonomous vehicle <NUM>).

Status indication system <NUM> includes an interface system (e.g., interface system202) configured to receive perception information (e.g., perception information <NUM>) concerning one or more objects (e.g., objects <NUM>, <NUM>, <NUM>, <NUM>) proximate the moveable vehicle (e.g., autonomous vehicle <NUM>). The one or more objects (e.g., objects <NUM>, <NUM>, <NUM>, <NUM>) proximate the moveable vehicle (e.g., autonomous vehicle 10include one or more of:.

As discussed above, sensors <NUM> within autonomous vehicle <NUM> monitors the environment in which autonomous vehicle <NUM> is operating, wherein sensors <NUM> provide sensor data <NUM> to ECUs <NUM> that may be processed to determine the manner in which autonomous vehicle <NUM> should operate. Interface system <NUM> is configured to interface with sensors <NUM> (generally) and one of more machine vision sensors (specifically) included within the moveable vehicle (e.g., autonomous vehicle <NUM>). Example of such machine vision sensors include but are not limited to LIDAR system.

As is known in the art, LIDAR is a method for determining ranges (variable distance) by targeting an object with a laser and measuring the time for the reflected light to return to the receiver. Lidar is also be used to make digital <NUM>-D representations of areas on the earth's surface and ocean bottom, due to differences in laser return times, and by varying laser wavelengths. It has terrestrial, airborne, and mobile applications. Lidar is an acronym of "light detection and ranging" or "laser imaging, detection, and ranging". Lidar sometimes is called <NUM>-D laser scanning, a special combination of a <NUM>-D scanning and laser scanning. Lidar iscommonly used to make high-resolution maps, with applications in surveying, geodesy, geomatics, archaeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. The technology is also used in control and navigation for some autonomous cars.

Status indication system <NUM> includes a processing system (e.g., processing system <NUM>) configured to process the perception information (e.g., perception information <NUM>) to generate proximate object display information (e.g., proximate object display information <NUM>). Processing system <NUM> is configured in various fashions. One example of processing system <NUM> includes a stand-alone processing system that includes one or more processors (not shown) and one or more memory architectures (not shown). Another example of processing system <NUM> includes a portion of ECUs <NUM>. Processing system <NUM> is coupled to a storage device (e.g., storage device <NUM>). Examples of storagedevice <NUM> include but are not limited to: a hard disk drive; a RAID device; a random- access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices. Status indication system <NUM> includes a visual display system (e.g., visual display system <NUM>) configured to render the proximate object display information (e.g., proximate object display information <NUM>). As will be discussed below in greater detail, the visual display system (e.g., visual display system <NUM>) is configured to convey the status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people proximate autonomous vehicle <NUM>.

Additionally, the interface system (e.g., interface system <NUM>) is configured to receive moveable vehicle status information (e.g., moveable vehicle status information <NUM>). As stated above, sensors <NUM> within autonomous vehicle <NUM> monitors the environment in which autonomous vehicle <NUM> is operating, wherein sensors <NUM> may provide sensor data <NUM> to ECUs <NUM>. Accordingly, the interface system (e.g., interface system <NUM>) obtains such moveable vehicle status information (e.g., moveable vehicle status information <NUM>) from ECUs <NUM>.

Accordingly, the moveable vehicle status information (e.g., moveable vehicle status information <NUM>) identifies one or more of the following:.

The processing system (e.g., processing system <NUM>) is configured to process the moveable vehicle status information (e.g., moveable vehicle status information <NUM>) to generate vehicle status display information (e.g., vehicle status display information <NUM>), wherein the visual display system (e.g., visual display system <NUM>) is configured to render the vehicle status display information (e.g., vehicle status display information <NUM>). As will be discussed below in greater detail, the visual display system (e.g., visual display system <NUM>) is configured to convey the status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people proximate autonomous vehicle <NUM>.

The visual display system (e.g., visual display system <NUM>) is configured to bemounted on a roof (e.g., roof <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>). The visual display system (e.g., visual display system <NUM>) is configured in various fashions, all of which are considered to be within the scope of this invention.

Cylindrical: The visual display system (e.g., visual display system <NUM>) can be a cylindrical visual display system (as shown in <FIG>). When visual display system <NUM> is configured in a cylindrical fashion, visual display system <NUM> includes: an illuminated portion (e.g., illumined portion <NUM>); and a non-illuminated portion (non-illuminated portion <NUM>) positioned between the illuminated portion (e.g., illumined portion <NUM>) and the roof (e.g.,roof <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>), thus providing a "hovering" appearance with respect to the roof (e.g., roof <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>).

Disk-Shaped: The visual display system (e.g., visual display system <NUM>) can be a disk-shaped visual display system (as shown in <FIG>). When visual display system <NUM> is configured in a disk-shaped fashion, visual display system <NUM> is generally illuminated proximate the roof (e.g., roof <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>), thus providing a lower profile that may be desirable when used on higher profile vehicles (such as SUVs and vans).

Regardless of configuration, the visual display system (e.g., visual display system <NUM>) is a <NUM> degree visual display system, thus conveying the status information concerning autonomous vehicle <NUM> in a visual fashion regardless of where the people are positioned proximate autonomous vehicle <NUM>.

While the visual display system (e.g., visual display system <NUM>) is described aboveas being mounted on a roof (e.g., roof <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>), this is for illustrative purposes only and is not intended to be a limitation of this invention, as other configurations are possible and are considered to be within the scope of this invention. For example, the visual display system (e.g., visual display system <NUM>) is integrated into the moveable vehicle (e.g., autonomous vehicle <NUM>) in various fashions. Accordingly, the visual display system (e.g., visual display system <NUM>) is a portion of (or incorporated within) a window (e.g., window <NUM>) of the moveable vehicle (e.g., autonomous vehicle <NUM>).

As discussed above, the visual display system (e.g., visual display system <NUM>) is configured to convey status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people proximate autonomous vehicle <NUM>.

Referring also to <FIG> and in order to convey such status information in such a visual fashion, processing system <NUM> executes status indication process <NUM>. As discussedabove, processing system <NUM> is configured in various fashions, examples of which may include but are not limited to:.

The instruction sets and subroutines of status indication process <NUM>, which is stored on storage device <NUM> coupled to ECUs <NUM>, is executed by one or more processors (not shown) and one or more memory architectures (not shown) included within ECUs <NUM>. Examples of storage device <NUM> includes but are not limited to: a hard disk drive; a RAID device; a random-access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

Status indication process <NUM> is executed on a single ECU or is executedcollaboratively across multiple ECUs. For example, status indication process <NUM> is executed solely by autonomy control unit <NUM>, vehicle control unit <NUM> or vehicle control unit <NUM>. Alternatively, status indication process <NUM> is executed collaboratively across the combination of autonomy control unit <NUM>, vehicle control unit <NUM> and vehicle control unit <NUM>. Accordingly and in the latter configuration, in the event of a failure of one of autonomy controlunit <NUM>, vehicle control unit <NUM> or vehicle control unit <NUM>, the surviving control unit(s) continues to execute status indication process <NUM>.

Status indication process <NUM> may monitor <NUM> one or more machine vision sensors(e.g., sensors <NUM> generally and a LIDAR sensor specifically) to obtain perception information (e.g., perception information <NUM>) concerning one or more pedestrians (e.g., objects <NUM>, <NUM>, <NUM>) proximate an autonomous vehicle (e.g., autonomous vehicle <NUM>). Perception information <NUM> is a three-dimensional image that generally identifies all objects in geographic proximity of autonomous vehicle <NUM>.

Status indication process <NUM> identifies <NUM> one or more humanoid shapes within the perception information (e.g., perception information <NUM>), thus defining one or more detected humanoid shapes. When identifying <NUM> one or more humanoid shapes within the perception information (e.g., perception information <NUM>), status indication process <NUM> compares <NUM> one or more defined humanoid shapes (e.g., defined humanoid shapes <NUM>) to one or more unidentified objects within the perception information (e.g., perception information <NUM>) to identify one or more humanoid shapes within the perception information (e.g., perception information <NUM>).

Defined humanoid shapes <NUM> are manually-defined (e.g., via a designer / programmer of status indication process <NUM>) and/or automatically-defined (e.g., via Artificial Intelligence/ Machine Learning (AI/ML) in a fashion similar to the manner in which AI/ML can identify human faces within photographs). Generally speaking, perception information <NUM> is a three-dimensional image that generally identifies a plurality of unidentified objects that are in geographic proximity of autonomous vehicle <NUM>. Accordingly, status indication process <NUM> may compare <NUM> defined humanoid shapes <NUM> to one or more unidentified objects within the perception information (e.g., perception information <NUM>) to identify one or more humanoid shapes within the perception information (e.g., perception information <NUM>).

Once the humanoid shapes are identified <NUM> within perception information <NUM>, status indication process <NUM> generates <NUM> proximate object display information (e.g., proximate object display information <NUM>) that locates the one or more detected humanoid shapes with respect to the autonomous vehicle (e.g., autonomous vehicle <NUM>). Status indication process <NUM> then renders <NUM> the proximate object display information (e.g., proximate object display information <NUM>) on a visual display system (e.g., visual display system <NUM>), thus confirming the perception of the one or more pedestrians (e.g., objects <NUM>, <NUM>, <NUM>) by the autonomous vehicle (e.g., autonomous vehicle <NUM>).

For example, <FIG> illustrates a situation in which two pedestrians (e.g., pedestrians <NUM>, <NUM>) are standing stationary proximate the side of autonomous vehicle <NUM>. Accordingly, proximate object display information <NUM> locates the one or more detected humanoid shapes (e.g., pedestrians <NUM>, <NUM>) with respect to autonomous vehicle <NUM>. Further, visual display system <NUM> of status indication system <NUM> conveys status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people (e.g., pedestrians <NUM>, <NUM>) proximate autonomous vehicle <NUM>. Accordingly, visual display system <NUM> renders a first indicator (e.g., indicator <NUM>) that is pointing to (or corresponds with) pedestrian <NUM>. As pedestrian <NUM> is stationary, indicator <NUM> is stationary within visual display system <NUM>. Further, visual display system <NUM> renders a second indicator (e.g., indicator <NUM>) that is pointing to (or corresponds with) pedestrian <NUM>. As pedestrian 302is stationary, indicator <NUM> may be stationary within visual display system <NUM>. Further, <FIG> illustrates a situation in which one pedestrian (e.g., pedestrian <NUM>, <NUM>) is standing stationary proximate the front of autonomous vehicle <NUM>. Accordingly, proximate object display information <NUM> locates the one or more detected humanoid shapes (e.g., pedestrian <NUM>) with respect to autonomous vehicle <NUM>. Further, visual display system <NUM> of status indication system <NUM> conveys status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people (e.g., pedestrian <NUM>) proximate autonomous vehicle <NUM>. Accordingly, visual display system <NUM> renders an indicator (e.g., indicator <NUM>) that is pointing to (or corresponds with) pedestrian <NUM>. As pedestrian <NUM> is stationary, indicator <NUM> is stationary within visual display system <NUM>. Further, as pedestrian <NUM> is in the path of (i.e., obstructing) autonomous vehicle <NUM>, indicator <NUM> provides instructions to pedestrian <NUM> in the form of e.g., the leftward-facing arrow that is asking pedestrian <NUM> to move to the left and out of the path of autonomous vehicle <NUM>.

The proximate object display information (e.g., proximate object display information <NUM>) includes dynamic proximate object display information (e.g., proximate object display information <NUM>) that changes as the location of the one or more detected humanoid shapes changes with respect to the autonomous vehicle (e.g., autonomous vehicle <NUM>). Accordingly and when rendering <NUM> the proximate object display information (e.g., proximate object display information <NUM>) on a visual display system (e.g., visual display system <NUM>), status indication process <NUM> renders <NUM> the dynamic proximate object display information (e.g., proximate object display information <NUM>) on the visual display system (e.g., visual display system <NUM>), thus dynamically confirming the perception of the one or more pedestrians (e.g., objects <NUM>, <NUM>, <NUM>) by the autonomous vehicle (e.g., autonomous vehicle <NUM>).

For example, <FIG> illustrates a situation in which two pedestrians (e.g., pedestrians <NUM>, <NUM>) are walking in front of autonomous vehicle <NUM>. Accordingly, proximate object display information <NUM> locates the one or more detected humanoid shapes (e.g., pedestrians <NUM>, <NUM>) with respect to autonomous vehicle <NUM>. Further, visual display system <NUM> of status indication system <NUM> conveys status information concerning autonomous vehicle <NUM> in a visual fashion that is easily understandable to people (e.g., pedestrians <NUM>, <NUM>) proximate autonomous vehicle <NUM>. Accordingly, visual display system <NUM> renders a first indicator (e.g., indicator <NUM>) that is pointing to (or corresponds with) pedestrian <NUM>. As pedestrian <NUM> is moving right to left, indicator <NUM> also moves right to left within visual display system <NUM>. Further, visual display system <NUM> renders a second indicator (e.g., indicator <NUM>) that is pointing to (or corresponds with) pedestrian <NUM>. As pedestrian <NUM> is moving left to right, indicator <NUM> also moves left to right within visual display system <NUM>.

In a fashion similar to the manner in which status indication process <NUM> tracks pedestrians proximate an autonomous vehicle (e.g., autonomous vehicle <NUM>), status indication process <NUM> also tracks vehicles proximate an autonomous vehicle (e.g., autonomous vehicle10). For example and referring also to <FIG>, status indication process <NUM> monitors350 one or more machine vision sensors (e.g., sensors <NUM> generally and a LIDAR sensor specifically) to obtain perception information (e.g., perception information <NUM>) concerning one or more third-party vehicles (e.g., object <NUM>) proximate an autonomous vehicle (e.g., autonomous vehicle <NUM>).

Status indication process <NUM> identifies <NUM> one or more vehicle shapes within the perception information (e.g., perception information <NUM>), thus defining one or more detected vehicles shapes. When identifying <NUM> one or more vehicle shapes within the perception information (e.g., perception information <NUM>), status indication process <NUM> compares <NUM> one or more defined vehicle shapes (e.g., defined vehicle shapes <NUM>) to one or more unidentified objects within the perception information (e.g., perception information <NUM>) to identify one or more vehicle shapes within the perception information (e.g., perception information <NUM>).

Defined vehicle shapes <NUM> are manually-defined (e.g., via a designer / programmer of status indication process <NUM>) and/or automatically-defined (e.g., via Artificial Intelligence/ Machine Learning (AI/ML) in a fashion similar to the manner in which AI/ML can identify vehicles within photographs). Generally speaking, perception information <NUM> is a three-dimensional image that generally identifies a plurality of unidentified objects that are in geographic proximity of autonomous vehicle <NUM>. Accordingly, status indication process <NUM> compares <NUM> defined vehicle shapes <NUM> to one or more unidentified objects within the perception information (e.g., perception information <NUM>) to identify one or more vehicle shapes within the perception information (e.g., perception information <NUM>).

Once the vehicle shapes are identified <NUM> within perception information <NUM>, status indication process <NUM> generates <NUM> proximate object display information (e.g., proximateobject display information <NUM>) that locates the one or more detected vehicles shapes with respect to the autonomous vehicle (e.g., autonomous vehicle <NUM>). Status indication process <NUM> then renders <NUM> the proximate object display information (e.g., proximate object display information <NUM>) on a visual display system (e.g., visual display system <NUM>), thus confirming the perception of the one or more third-party vehicles (e.g., object <NUM>) by the autonomous vehicle (e.g., autonomous vehicle <NUM>).

The proximate object display information (e.g., proximate object displayinformation <NUM>) includes dynamic proximate object display information (e.g., proximateobject display information <NUM>) that changes as the location of the one or more detected vehicle shapes changes with respect to the autonomous vehicle (e.g., autonomous vehicle <NUM>). Accordingly and when rendering <NUM> the proximate object display information (e.g., proximate object display information <NUM>) on a visual display system (e.g., visual display system <NUM>),status indication process <NUM> renders <NUM> the dynamic proximate object display information (e.g., proximate object display information <NUM>) on the visual display system (e.g.,visual display system <NUM>), thus dynamically confirming the perception of the one or more third-party vehicles (e.g., object <NUM>) by the autonomous vehicle (e.g., autonomous vehicle <NUM>).

As shown on <FIG>, visual display system <NUM> renders various indicators (e.g., indicators <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), wherein each of these indicators (e.g.,indicators <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) is shown to include e.g., a "stick figure" that indicates that the indicator (e.g., indicators <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) is pointing to (or corresponds with) a pedestrian (e.g., pedestrians <NUM>, <NUM>, <NUM>, <NUM>, <NUM>). Accordingly and referring also to <FIG>, when status indication process <NUM> is tracking vehicles proximate autonomous vehicle <NUM>, visual display system <NUM> renders various indicators (e.g., indicator <NUM>) that includes e.g., a "car figure" that is indicative of the indicator (e.g., indicator <NUM>) pointing to (or corresponds with) a vehicle (e.g., third-party vehicle <NUM>).

As discussed above, the interface system (e.g., interface system <NUM>) is configured to receive moveable vehicle status information (e.g., moveable vehicle status information <NUM>), wherein this moveable vehicle status information (e.g., moveable vehicle status information <NUM>) identifies one or more of the following:.

The processing system (e.g., processing system <NUM>) is configured to process the moveable vehicle status information (e.g., moveable vehicle status information <NUM>) to generate vehicle status display information (e.g., vehicle status display information <NUM>), wherein the visual display system (e.g., visual display system <NUM>) is configured to render the vehicle status display information (e.g., vehicle status display information <NUM>).

Accordingly and as shown in <FIG>, proximate object portion <NUM> of visual display system <NUM> is configured to track pedestrians proximate an autonomous vehicle (e.g., autonomous vehicle <NUM>). Further, status portion <NUM> of visual display system <NUM> is configured to render vehicle status display information <NUM>. For example, status portion <NUM> may include:.

Additionally and as shown in <FIG>, proximate object portion <NUM> of visual display system <NUM> is configured to track third-party vehicles proximate an autonomous vehicle (e.g., autonomous vehicle <NUM>). Again, status portion <NUM> of visual display system <NUM> is configured to render vehicle status display information <NUM> (as described above).

As will be appreciated by one skilled in the art, the present invention may be embodied as a method, a system, or a computer program product. Accordingly, the present invention takes the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that all generally be referred to herein as a "circuit," "module" or "system. " Furthermore, the present invention takes the form of a computer program product on a computer-usable storage medium having computer-usable program code embodied in the medium.

Any suitable computer usable or computer readable medium is utilized. The computer-usable or computer-readable medium is, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a non-exhaustive list) of the computer-readable medium includes the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a transmission media such as those supporting the Internet or an intranet, or a magnetic storage device. The computer-usable or computer-readable medium also is paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of this document, a computer-usable or computer-readable medium is any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-usable medium includes a propagated data signal with the computer-usable program code embodied therewith, either in baseband or as part of a carrier wave. The computer usable program code is transmitted using any appropriate medium, including but not limited to the Internet, wireline, optical fiber cable, RF, etc..

Computer program code for carrying out operations of the present invention is written in an object oriented programming language such as Java, Smalltalk, C++ or the like. However, the computer program code for carrying out operations of the present invention is be written in conventional procedural programming languages, such as the"C" programming language or similar programming languages. The program code executes entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer is connected to the user's computer through a local area network I a wide area network I the Internet (e.g., network14).

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, are implemented by computer program instructions. These computer program instructions is provided to a processor of a general purpose computer/ special purpose computer / other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified m the flowchart and/or block diagram block or blocks.

These computer program instructions are also be stored in a computer-readable memory that directs a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions are also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented processsuch that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrates the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams represents a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block occurs out of the order noted in the figures. For example, two blocks shown in succession are, in fact, be executed substantially concurrently, or the blocks are sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, are implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Claim 1:
A computer-implemented method, to be executed on a computing device, the computer-implemented method for an autonomous vehicle (<NUM>) comprising:
monitoring one or more machine vision sensors (<NUM>) included within the autonomous vehicle (<NUM>) to obtain perception information concerning one or more pedestrians (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) proximate the autonomous vehicle (<NUM>);
identifying one or more humanoid shapes (<NUM>) within the perception information, thus defining one or more detected humanoid shapes (<NUM>);
generating proximate object display information (<NUM>) that locates the one or more detected humanoid shapes (<NUM>) with respect to the autonomous vehicle; and
rendering the proximate object display information (<NUM>) on a visual display system (<NUM>) of the autonomous vehicle, thus confirming the perception of the one or more pedestrians (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) by the autonomous vehicle (<NUM>),
wherein the rendering on the visual display system (<NUM>) includes rendering at least an indicator (<NUM>) that is pointing to a stationary pedestrian (<NUM>), wherein at least the indicator (<NUM>) is stationary within visual display system (<NUM>), and wherein the rendering includes rendering instructions on the visual display system (<NUM>) of the autonomous vehicle (<NUM>), as a pedestrian (<NUM>) is in the path of the autonomous vehicle (<NUM>) is detected, an indicator (<NUM>) providing instructions to pedestrian (<NUM>) in the form of an arrow that is asking pedestrian (<NUM>) to move out of the path of autonomous vehicle (<NUM>), and wherein the rendering on the visual display system (<NUM>) includes rendering a first indicator (<NUM>) that is pointing to or corresponds with a first pedestrian (<NUM>) and as the first pedestrian (<NUM>) is moving right to left, the first indicator (<NUM>) also moves right to left within visual display system (<NUM>) and further, the rendering includes rendering a second indicator (<NUM>) that is pointing to or corresponds with a second pedestrian (<NUM>) and as the second pedestrian (<NUM>) is moving left to right, second indicator (<NUM>) also moves left to right within visual display system (<NUM>).