Displaying next action of an autonomous vehicle

Systems, methods and computer program products that facilitate displaying next action of an autonomous vehicle. A system can include a memory and a processor that executes computer executable components. The computer executable components can include: an analysis component that determines or infers next action of an autonomous vehicle and a display component that generates a graphical user interface that visually conveys the next action, wherein the graphical user interface comprises a shape that dynamically morphs to visually represent the next action.

TECHNICAL FIELD

Embodiments disclosed and claimed herein relate to techniques that facilitate displaying next action of an autonomous vehicle.

BACKGROUND

Passengers in an autonomous vehicle can experience motion sickness, anxiety or other discomforts associated with not knowing next navigation or driving acts made by an autonomous vehicle. By displaying next action of an autonomous vehicle (AV) in an easy to understand graphical image, the passengers can quickly be informed of the next actions or movements being made by the autonomous vehicle. This can avoid surprises, reduce anxiety and reduce need for passengers to observe the autonomous vehicle's surroundings in order to understand the AV's movements or next action. Conventional systems generally utilize static images or a combination of multiple images to convey the next action of an autonomous vehicle. The static nature of such systems often results in minimal relief to passengers.

SUMMARY

Graphical user interfaces used by conventional systems to inform passengers of next actions or movements being made by an autonomous vehicle can be improved by employing a shape that dynamically morphs to visually represent the next action of the AV as opposed to the static images used by conventional systems. A shape that dynamically morphs to visually represent the next action can convey information about next action more effectively with just a glance by a passenger of the autonomous vehicle. It also conveys more information such as relative changes in degree associated with factors such as speed, acceleration, direction and turn radius and concurrent changes such as changes in direction and speed.

In accordance with an embodiment, a system comprises: a memory and a processor that executes computer executable components. The computer executable components can include an analysis component that determines or infers next action of an autonomous vehicle, and a display component that generates a graphical user interface that visually conveys the next action, wherein the graphical user interface comprises a shape that dynamically morphs to visually represent the next action.

In some embodiments, elements described in connection with the disclosed systems can be embodied in different forms such as a computer-implemented method, a computer program product, or another form.

DETAILED DESCRIPTION

The following detailed description is merely illustrative and is not intended to limit embodiments and/or application or uses of embodiments. Furthermore, there is no intention to be bound by any expressed or implied information presented in the preceding Summary section or in the Detailed Description section.

Embodiments described herein include systems, methods, and computer program products that facilitate displaying next action of an autonomous vehicle. Instead of relying upon static images or multiple images to convey next action of an autonomous vehicle to passengers of the AV, a graphical user interface (GUI) uses a shape that dynamically morphs to visually represent the next action of the vehicle. A geometric figure that dynamically morphs to visually represent the next action can convey information about next action more effectively in a glanceable manner to a passenger of the autonomous vehicle. It also conveys more information such as relative changes in degree associated with factors such as speed, acceleration, direction and turn radius and concurrent changes such as changes in direction and speed.

FIG.1illustrates a block diagram of an example, non-limiting system100that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. The system100includes a processor102that executes computer executable components stored in at least one memory104. The system100can further include a system bus106that can couple various components, including, but not limited to, an analysis component108and a display component110. The analysis component108determines or infers next action of an autonomous vehicle. The display component110generates a graphical user interface that visually conveys the next action, wherein the generated visualization comprises a shape that dynamically morphs to visually represent the next action of the AV. Since next actions (e.g., a hard turn, braking, acceleration, passing another vehicle . . . ) are not binary or static but rather a fluid set of acts across a continuum from a first state to an end state the generated visualization likewise dynamically morphs the shape in real-time to coincide with the next action across the continuum of sub-acts.

In certain embodiments, the analysis component108can determine or infer next action of the autonomous vehicle. For example, the next action can include, for example: a turn, braking, lane change or change in direction, speed, acceleration, deceleration or degree of a turn and the like. The analysis component108can utilize data received from systems and devices within the autonomous vehicle as well as extrinsic information (e.g., weather, global positioning system (GPS) data . . . ) to determine or infer the next action. In one example, the analysis component108can utilize a route selected by a navigation system of the autonomous vehicle. The selected route can serve as a baseline of actions to be taken by the autonomous vehicle, and the analysis component108can utilize the GPS of the autonomous vehicle to determine location of the AV along a selected route. The analysis component108can also utilize adjustments made to the route by the navigation system. The analysis component108can utilize data received from other systems and devices of the autonomous vehicle in determining or inferring next action as the autonomous vehicle proceeds on the selected route. For example, the analysis component108can utilize data received from machine vision systems and devices in order to detect traffic signals. Also, the analysis component108can utilize data received from machine vision systems and devices in order to detect presence of pedestrians, cyclists, obstructions or other vehicles that can affect the next action of the autonomous vehicle. In one example, the presence of one or more vehicles or cyclists in front of the autonomous vehicle may cause the autonomous vehicle to decelerate as it proceeds on a straight road. In another example, an autonomous vehicle stopped at a red light can wait to begin a right turn that will be the autonomous vehicle's next action until a pedestrian finishes crossing the street in front of the autonomous vehicle. Information from other vehicle and devices, e.g., for crash avoidance, traffic, speed traps . . . can be utilized in connection with determining or inferring next action to be taken by the AV and generating a dynamic visualization to convey the upcoming action.

In another example, the analysis component108can utilize traffic information received wirelessly by the autonomous vehicle. For example, information associated with an accident on a selected route can be utilized by the analysis component108to determine that that autonomous vehicle will be slowing down or that another route can be selected by the navigation system.

In another example, the analysis component108can utilize information received wirelessly by the autonomous vehicle from other vehicles associated with location, movement and anticipated movement of each such other vehicle. For example, if a vehicle traveling in front of the autonomous vehicle in the same direction transmits information to the autonomous vehicle that the vehicle will be slowing down to make a right turn at the next intersection, the analysis component108can utilize such information to determine or infer that the next action of the autonomous vehicle will be to slow down or to change lanes as the vehicle in front of it slows down to make the turn depending on other factors such as proximity of other vehicles.

In another example, the analysis component108can utilize data received from devices such as sensors that identify road surface or road conditions such as a pothole or other road hazard that can possibly cause damage to the autonomous vehicle or affect safety or comfort of passengers of the autonomous vehicle. In this example, the autonomous vehicle can slow down or swerve to avoid the pothole or other road hazard depending on other factors such as the proximity of other vehicles. In another example, the analysis component108can determine that the vehicle will reduce speed on a gravel road. These determined or inferred actions can be visually represented by the dynamic morphing shape.

In another example, the analysis component108can utilize data associated with weather. In one example, the analysis component108can utilize data received from devices in the autonomous vehicle such as sensors (e.g., thermometer, barometer, tire pressure, moisture, oil, debris or ice detectors, vehicle operation sensors . . . ) that identify real-time weather conditions. In addition, the analysis component108can determine the extent to which identified weather conditions can affect road conditions or operation of the autonomous vehicle. For example, if the autonomous vehicle is traveling on a curvy, mountain road and a rainstorm begins, the analysis component108can determine or infer that the autonomous vehicle will reduce speed. In another example, the analysis component108can utilize data received from wirelessly from services that provide real-time weather information or weather forecasts. For example, the analysis component108can determine or infer that the autonomous vehicle will be reducing speed if weather data indicates that snow will begin falling on autonomous vehicle's current route.

In another example, the analysis component108can utilize data received from systems or devices in the autonomous vehicle that monitor the operation or required maintenance of systems, devices or components of the autonomous vehicle. For example, if air pressure of one or more tires of the autonomous vehicle falls below a certain threshold, the analysis component108can determine or infer that the autonomous vehicle will reduce speed when making certain turns.

In another example, the analysis component108can utilize data received wirelessly from third party sources associated with events such as concerts, sporting events, festivals and the like that can affect the navigation selections made by the autonomous vehicle's navigation system due to anticipated traffic. In another example, the analysis component108can utilize data received wirelessly from third party sources associated with road construction or maintenance.

In an embodiment, the analysis component108can utilize data received wirelessly from an external traffic coordination system in order to determine or infer next action of the autonomous vehicle. For example, an external traffic coordination system can coordinate movement of multiple autonomous vehicles in a defined geographic area in order to optimize movement of each AV in reaching respective destination within the context of other vehicles.

In certain embodiments, the display component110can generates a graphical user interface that visually conveys the next action, wherein the graphical user interface comprises a shape that dynamically morphs to visually represent the next action. For example, one shape can morph into different shapes in order to represent next action such as a turn, braking or lane change or a change in direction, speed, acceleration, deceleration or the degree of a turn and the like. For example, as an autonomous vehicle approaches right turn, the shape generated by the display component110can dynamically morph into another shape to indicate next actions associated with changes in speed, direction and turn radius as the autonomous vehicle proceeds through the turn. Unlike static images, a shape that dynamically morphs into another shape can also convey next actions associated with changes in degree in speed, acceleration, deceleration, direction and turn radius in real time. In one example, as an autonomous vehicle decelerates into a turn to the right and then accelerates as the autonomous vehicle completes the turn and continues in substantially a straight direction, all changes associated with this movement by the autonomous vehicle can be visually represented by a shape that dynamically morphs to visually represent the next action. A passenger can quickly understand the next actions with just a glance at the morphing shape.

In certain embodiments, a shape generated by the display component110can comprise one or more types of shapes or combinations of shapes such as lines, angles, curves, polygons, circles, ellipses, non-geometrical shapes and the like. For example, a shape generated by the display component110can comprise an arrow in a form commonly used on street signs to indicate direction and can morph from a straight arrow to a curved arrow to indicate the autonomous vehicle will be turning. In another example, shapes of one or more types can morph into other types of shapes. For example, a shape generated by the display component110can comprise an arrow in a form commonly used on street signs to indicate forward movement and can morph into the form of a stop sign to indicate the autonomous vehicle will be coming to a stop.

In another example, the shape displayed by the display component110can comprise a horizontal line to visually represent a stationary autonomous vehicle. In this example, the horizontal line can dynamically morph to form a directional angle to visually represent the next action of the autonomous vehicle (seeFIGS.2A-2Dbelow). The directional angle can take the form of two substantially straight lines connecting at a point. The distance of the connection point of the two substantially straight lines forming the directional angle from the original horizontal line can visually represent speed. As the distance of the connection point from the original horizontal line increases, it conveys an increase in speed. The rate of change can visually represent acceleration or deceleration. As the connection point of the two substantially straight lines moves horizontally to the left or the right, the directional angle changes to convey the change in direction of the autonomous vehicle. As the connection point moves to the far left or far right, the directional angle can convey the extent to which the autonomous vehicle is approaching its maximum turn radius. Speed, acceleration, deceleration, direction and turn radius can be conveyed and quickly understood with a glance at the graphical user interface generated by the display component110as the connection point moves, causing the directional angle to dynamically morph to indicate the next action of the autonomous vehicle. A return of the directional angle to the horizontal straight line can convey that the autonomous vehicle has stopped moving. Thus, with a horizontal line dynamically morphing to form a directional angle, the display component110can convey the next action of an autonomous vehicle the simultaneous degree of change of multiple variables. For example, as an autonomous vehicle approaches a turn by slowing down, then gradually turning the vehicle and then accelerating through the end of the turn, all of the changes in speed, acceleration, deceleration, direction and turn radius can be conveyed and quickly understood with a glance at the dynamically morphing directional angle generated by the display component110.

In an embodiment, the shape displayed by the display component110can utilize color to visually represent the next action. For example, if the shape displayed by the display component110comprises a horizontal line that dynamically morphs to form a directional angle to visually represent the next action, the display component110can change the color of the directional angle to visually represent additional information association with next action. In one example, if an autonomous vehicle is slowing down due to factors such as bad weather or other safety factors, the color of the directional angle displayed by the display component110can change from white to red. In another example, the color of the directional angle displayed by the display component110can change from white to red when roads are slippery due to factors such as ice, snow, sleet or the like.

The shape displayed by the display component110can utilize line types to visually represent the next action. For example, substantially straight lines comprising a directional angle can dynamically morph into jagged lines to convey a bumpy road. In another example, substantially straight lines comprising a directional angle can dynamically morph into wavy lines to convey a slippery road.

The shape displayed by the display component110can comprise an arrow in shapes typically used in street signs that dynamically morphs to visually represent the next action. For example, an arrow pointing up can convey forward motion by the autonomous vehicle and an arrow curving to the right can convey a right turn. The length of the arrow can convey speed. In this example, the length of the arrow can dynamically morph as the autonomous vehicle begins to increase speed and can dynamically shorten as the autonomous vehicle begins to decrease speed. As the straight arrow dynamically morphs into an arrow curving to the right it visually represents the autonomous vehicle turning to the right.

In various embodiments, the display component110can include settings that enable a passenger to adjust the graphical user interface for characteristics such as type of shape, color, brightness, contrast and the like. The graphical user interface generated by the display component110can convey a notification to a driver of the autonomous vehicle to take control of the autonomous vehicle. The display component110can visually represent the next action as one or more projected images. For example, the shape visually representing the next action can be projected into the field of vision of a passenger in the driver's seat as a head up display.

In another embodiment, the display component110can visually represent the next action as one or more three-dimensional images. For example, if the shape used by the display component110can morph into the shape of a directional angle pointing upward to convey forward motion of the autonomous vehicle, the shape can be visually represented as a three-dimensional image with the directional angle pointing forward in a three-dimensional setting as opposed to pointing upward on a flat screen.

In yet another embodiment, the display component110can visually represent the next action in an augmented realty environment. For example, the display component110can overlay a geometric image onto the field of vision of a passenger of an autonomous vehicle. In one example, the display component110can generate a line that morphs into a directional angle in the passenger's field of vision with the directional angle overlaid onto the road and upcoming turns in the passenger's field of vision.

FIGS.2A-2Drespectively illustrate an example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIGS.2A-2Ddepict examples of a shape that can be displayed in a graphical user interface generated by the display component110that visually conveys the next action of an autonomous vehicle by dynamically morphing to form a directional angle to visually represent the next action.FIG.2Adepicts a shape202ain the form of a substantially straight line positioned horizontally in the graphical user interface and visually represents the autonomous vehicle being at rest.FIG.2Bdepicts shape202b, in the form of an angle with vertex or apex206above the position of the horizontal line of shape202ainFIG.2A, with the opposite ends of line208that forms the angle located at the position of the graphical user interface of the horizontal line of shape202ainFIG.2A. In this example, the shape202bis in the form of a directional angle pointing upward and visually represents the autonomous vehicle moving forward in substantially a straight line.FIG.2Cdepicts shape202cin the form of an inverted angle with apex (or vertex)212below the position of the horizontal line of shape202ainFIG.2A, with the opposite ends line214that forms the angle202clocated at the position of the graphical user interface of the horizontal line of shape202ainFIG.2A. In this example, the shape202cis in the form of a directional angle pointing downward and visually represents the autonomous vehicle moving in reverse in substantially a straight line.FIG.2Ddepicts a shape202din the form of a directional angle with vertex or apex218above the position of the horizontal line of shape202ainFIG.2A, with the opposite ends of line220that forms the angle located at the position of the graphical user interface of the horizontal line of shape202ainFIG.2A. In this example, however, the shape202dis in the form of a directional angle pointing upward and to the right and visually represents the autonomous vehicle moving forward and turning to the right. Passengers of the autonomous vehicle will see the shape202ain the form of a straight, horizontal line when the autonomous vehicle is stationary. As the autonomous vehicle is about to move forward in substantially a straight line, the shape202a(FIG.2A) will morph into the form of a directional angle pointing upward of shape202b(FIG.2B). As the autonomous vehicle is about to make a turn to the right, the shape202bwill morph into the form of a directional angle pointing upward and to the right of shape202d(FIG.2D). If the autonomous vehicle stops and then begins to move in reverse in substantially a straight line, passengers of the autonomous vehicle will see a shape in the form of substantially a straight line of shape202a(FIG.2A) that morphs into the form of a directional angle pointing downward of shape202c(FIG.2C) and visually represents the autonomous vehicle moving in reverse in substantially a straight line.

FIGS.3A-3Drespectively illustrate another example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIGS.3A-3Ddepict examples of a shape that can be displayed in a graphical user interface generated by the display component110that visually conveys the next action of an autonomous vehicle by dynamically morphing to form a directional angle to visually represent the next action.FIG.3Aillustrates a shape302ain the form of a substantially straight line positioned horizontally in the graphical user interface and visually represents the autonomous vehicle being at rest.FIG.3Bdepicts a shape302bin the form of an angle with vertex or apex306above the position of the horizontal line of shape302ainFIG.3A, with the opposite ends of line308that forms the angle302blocated at the position of the graphical user interface of the horizontal line of shape302ainFIG.3A. In this example, the shape302bis in the form of a directional angle pointing upward and visually represents the autonomous vehicle moving forward.FIG.3Cdepicts a shape302cin the form of an angle with vertex or apex312above the position of the vertex or apex306of shape302binFIG.3B, with the opposite ends of line314that forms the angle located at the position of the graphical user interface of the horizontal line of shape302ainFIG.3A. In this example, the shape302cis in the form of a directional angle pointing upward and visually represents the autonomous vehicle moving forward at a speed greater than the speed depicted in shape302b(FIG.3B).FIG.3Ddepicts a shape302din the form of an angle with vertex or apex318above the position of the vertex or apex312of shape302cinFIG.3C, with the opposite ends of line320that forms the angle located at the position of the graphical user interface of the horizontal line of shape302ainFIG.3A. In this example, the shape302dis in the form of a directional angle pointing upward and visually represents the autonomous vehicle moving forward at a speed greater than the speed depicted by shapes302bor302c. As the autonomous vehicle is about to move forward, the shape302a(FIG.3A) will morph into the form of a directional angle pointing upward of shape302b(FIG.3B). As the autonomous vehicle is about to increase speed as the autonomous vehicle continues to move forward in substantially a straight line, the shape302b(FIG.3B) will continue to morph (e.g., smaller angle) into figure the shape302c(FIG.3C). As the autonomous vehicle is about to further increase speed as it continues to move forward, the shape302c(FIG.3C) will continue to morph (at an even smaller angle) into the shape302d(FIG.3D). If the autonomous vehicle then decreases speed as it continues to move forward, the shape302d(FIG.3D) will morph back (at a larger angle) into shape302c(FIG.3C).

Thus, the shape (for example,302a,302b,302cand/or302d) dynamically morphs to represent in real-time next action and degree of action. As the AV speeds up, the shape points in the direction of an angle vertex and the angle becomes smaller, tighter, longer to reflect acceleration. As the AV slows down the shape dynamically morphs to widen the angle and will flatten to a line in a stopped or steady state position. Thus, the shape provides in a glanceable manner next actions and degree thereof.

FIGS.4A-4Drespectively illustrate yet another example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIGS.4A-4Ddepict examples of a shape that can be displayed in a graphical user interface generated by the display component110that visually conveys the next action of an autonomous vehicle by dynamically morphing to form a directional angle to visually represent the next action.FIG.4Adepicts a shape402ain the form of an angle with vertex or apex404. In this example, the shape402ais in the form of a directional angle pointing upward and visually represents the autonomous vehicle moving forward.FIG.4Bdepicts a shape402bin the form of an angle with vertex or apex408located to the right of the position of the vertex or apex404inFIG.4A. In this example, the shape402bis in the form of a directional angle pointing upward and to the right and visually represents the autonomous vehicle moving forward and turning to the right.FIG.4Cdepicts shape402cin the form of an angle with vertex or apex412located to the right of the position of the vertex or apex408inFIG.4B. In this example, the shape402cis in the form of a directional angle pointing upward and to the right and visually represents the autonomous vehicle moving forward and turning to the right with a sharper turn radius than the example depicted by shape402b(FIG.4B).FIG.4Ddepicts a shape402din the form of an angle with vertex or apex416located to the right of the position of the vertex or apex412inFIG.4C. In this example, the shape402dis in the form of a directional angle pointing upward and to the right and visually represents the autonomous vehicle moving forward and turning to the right with a sharper turn radius than the example depicted by shape402c(FIG.4C). As the autonomous vehicle is moving forward in substantially a straight line, the shape402a(FIG.4A) will morph into the form of a directional angle pointing upward and to the right of shape402b(FIG.4B) as the autonomous vehicle is about to begin turning right. As the AV is about to continue the turn to the right at a sharper turn angle, the shape402b(FIG.4B) will continue to morph into shape402c(FIG.4C). As the autonomous vehicle is about to further continue the turn to the right at a sharper turn angle, the shape402c(FIG.4C) will continue to morph into shape402d(FIG.4D). As the autonomous vehicle begins to complete the right turn and return to moving forward in substantially a straight line, the shape402d(FIG.4D) will morph into shape402c(FIG.4C), which will then morph into shape402b(FIG.4B), which will then morph into shape402a(FIG.4A).

FIGS.5A-5Drespectively illustrate yet another example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIGS.5A-5Ddepict examples of a shape in the form of an arrow in shapes typically used in street signs that can be displayed in a graphical user interface generated by the display component110that visually conveys the next action of an autonomous vehicle by dynamically morphing to alter the direction of the arrow.FIG.5Adepicts a shape502ain the form of an arrow pointing up504that visually represents forward motion by the autonomous vehicle.FIG.5Bdepicts a shape502bin the form of an arrow curving to the right508that visually represents the autonomous vehicle turning to the right.FIG.5Cdepicts shape502cin the form of an arrow curving downward512with the curve of the arrow forming a semi-circle that visually represents the autonomous vehicle continuing a turn in substantially the form of a semi-circle.FIG.5Ddepicts a shape502din the form of an arrow curving to the left516with the curve of the arrow forming a semi-circle that visually represents the autonomous vehicle continuing a turn in substantially the form of a semi-circle. As the autonomous vehicle is moving forward in substantially a straight line, the shape502a(FIG.5A) in the form of an arrow pointing upward will morph into the form of an arrow curving to the right502b(FIG.5B) as the autonomous vehicle is about to begin turning right. In this example, the autonomous vehicle is turning onto an on ramp to a freeway. As the autonomous vehicle continues to turn to the right through the on ramp, the shape502b(FIG.5B) will continue to morph into shape502c(FIG.5C). As the autonomous vehicle continues to turn to the right through the on ramp, the shape502c(FIG.5C) will continue to morph into shape502d(FIG.5D). As the autonomous vehicle begins to complete the turn through the on ramp onto the freeway, the shape502d(FIG.5D) will morph into shape502a(FIG.5A) as the autonomous vehicle is about to begin moving in substantially a forward direction.

FIG.6illustrates another example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIG.6depicts examples of a shape that can be displayed in a graphical user interface generated by the display component110that visually conveys the next action of an autonomous vehicle by dynamically morphing to form a directional angle to visually represent the next action.FIG.6depicts a shape602ain the form of a directional angle pointing upward to visually represent the autonomous vehicle moving forward in substantially a straight line.FIG.6also depicts shapes602b,602cand602din the form of directional angles pointing upward to visually represent the autonomous vehicle moving forward in substantially a straight line. In this example, an increase in the height of a directional angle pointing upward visually represents an increase in speed as the next action of the autonomous vehicle. As shape602amorphs into shape602b, then602cand then602d, the morphing shape602visually represents the autonomous vehicle increasing in speed as it moves forward in substantially a straight line. In this example, changes in color of the lines forming the directional angle will indicate changes in acceleration or deceleration as part of the next action of the autonomous vehicle. If the autonomous vehicle will be accelerating, the color of the directional angle will change to a shade of green. If the autonomous vehicle will be decelerating, the color of the directional angle will change to a shade of red. InFIG.6, the dashed lines represent a change in color from a default color to green or red. The longer dashes in a line represent a darker shade of either green or red. For example, shape602visually represents the autonomous vehicle moving forward. As the autonomous vehicle is about to accelerate as it moves forward, shape602amorphs into shape602bto visually indicate an increase in speed and changes color from the default color to a light shade of green to visually indicate acceleration. As the autonomous vehicle is about to continue to accelerate at a faster rate as it moves forward, shape602bmorphs into shape602cwhich becomes a darker shade of green, and then morphs into shape602dwhich becomes an even darker shade of green. If the autonomous vehicle reaches a top speed visually represented by shape602d, then the color of shape602dwould change from a dark shade of green to the default color to indicate that acceleration will cease. In another example, if analysis component108determines that traffic is much slower up ahead, the autonomous vehicle will begin to rapidly decelerate, and the rate of deceleration will decrease as the autonomous vehicle approaches the slower traffic ahead until it reaches a speed that is optimal for the slower traffic. As the autonomous vehicle is about to decelerate as it continues to move forward, shape602dmorphs into shape602cto visually indicate a decrease in speed and changes color from the default color to a dark shade of red to visually indicate rapid deceleration. As the autonomous vehicle is about to continue to decelerate at a slower rate as it continues to move forward, shape602cmorphs into shape602bwhich becomes a lighter shade of red, and then morphs into shape602awhich becomes an even lighter shade of red. Once the autonomous vehicle reaches the optimal speed for the slower traffic visually represented by shape602a, then the color of shape602awould change from a light shade of red to the default color to indicate that deceleration will cease.

FIGS.7A-7Erespectively illustrate yet another example of a non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.FIGS.7A-7Edepict examples of shapes that visually represent the next action of an autonomous vehicle on the left side of each figure.FIGS.7A-7Edepict examples of the position of the autonomous vehicle as an image on the right side of each figure corresponding to the shape that visually represents the next action. In these examples, the position of each shape representing next action will adjust to reflect North-South and East-West directions, with upward direction representing North, downward direction representing South, rightward direction representing East and leftward direction representing West.FIG.7Adepicts an autonomous vehicle700afacing North and at rest, and the shape702arepresenting next action is a horizontal line.FIG.7Bdepicts the next action of the autonomous vehicle700bwhich will be backing up and turning to the right. The image of the autonomous vehicle700bon the right depicts the position of the autonomous vehicle700bas it is proceeding through the reverse turn. The shape702brepresenting this next action is a curved directional arrow pointing downward and to the right. The horizontal line of shape702adepicted inFIG.7Amorphs into the curved directional arrow of shape702bdepicted inFIG.7Bas the autonomous vehicle is about to make the reverse turn.FIG.7Cdepicts the next action of the autonomous vehicle700cwhich will be stopping once it completes the reverse turn. Once stopped the vehicle will be facing West, and the shape702cvisually representing that next action is a vertical line. The curved directional arrow of shape702bdepicted inFIG.7Bmorphs into the vertical line of shape702cdepicted inFIG.7Cas the autonomous vehicle700cis about to stop.FIG.7Ddepicts the next action of the autonomous vehicle700dwhich will be moving forward and turning to the left. The image of the autonomous vehicle700don the right depicts the position of the autonomous vehicle700das it is proceeding through the left turn. The shape702drepresenting this next action is a curved directional arrow pointing to the left and downward. The vertical line of shape702cdepicted inFIG.7Cmorphs into the curved directional arrow of shape702ddepicted inFIG.7Das the autonomous vehicle700dis about to make the left turn.FIG.7Edepicts the next action of the autonomous vehicle700ewhich will be stopping once it completes the left turn. Once stopped the vehicle will be facing South, and the shape702evisually representing that next action is a horizontal line. The curved directional arrow of shape702ddepicted inFIG.7Dmorphs into the horizontal line of shape702edepicted inFIG.7Eas the autonomous vehicle700eis about to stop.

FIG.8illustrates a block diagram of another example, non-limiting system800that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. In certain embodiments, the system800includes an audio component802that can generate audio notifications of the next action. For example, the audio component802that can generate audio notifications of each next action conveyed by the display component110. In another example, the audio component802that can generate audio notifications of certain types of next action conveyed by the display component110. In this example, a passenger can determine which types of next action will generate an audio notification. For example, the audio component802can include settings that can be selected by a passenger that will cause audio notifications to be generated by the audio component802only for a next action that includes a sudden movement such as a sudden, sharp turn or sudden braking to avoid an obstruction.

In another embodiment, the audio component802can include settings that enable a passenger to adjust the audio notifications for characteristics such as language, volume, voice type and the like.

In another embodiment, the audio component802can generate audio notifications that can supplement information conveyed by the display component110. For example, the audio component802can generate audio notifications that explain why certain next actions will occur such as weather or traffic conditions.

FIG.9illustrates a block diagram of another example, non-limiting system900that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. In certain embodiments, the system900includes an integration component902that can integrate the system900with other visualization tools. For example, the system900can be integrated by the integration component902into a wireless mobile device such as a smartphone, tablet computer or the like in order to display the graphical user interface generated by the display component110. In another example, the integration component902can integrate the system900with lighting devices in the autonomous vehicle that can alert passengers when certain types of actions will be taken by the autonomous vehicle. For example, certain lights in the autonomous vehicle can change color or emit a flashing light when actions such as sharp turns are approaching.

FIG.10illustrates a block diagram of another example, non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. In certain embodiments, the system1000includes an artificial intelligence component1002that can facilitate the determination or inference of the next action of an autonomous vehicle by the analysis component108. For example, the artificial intelligence component1002can detect a pattern that pedestrians or cyclists are often present at a location on a common route taken by the autonomous vehicle at certain times on certain days. In this example, the pattern detected by the artificial intelligence component1002can facilitate an inference by the analysis component108that the autonomous vehicle will be slowing down at it approaches the location.

In another example, the artificial intelligence component1002can detect a pattern of how certain types of weather conditions can affect the operation of the autonomous vehicle on a particular road. For example, the artificial intelligence component1002can detect a pattern that rain makes a certain road very slippery that typically requires the autonomous vehicle to engage its traction control system. In this example, the pattern detected by the artificial intelligence component1002can facilitate an inference by the analysis component108that the autonomous vehicle will be moving more slowly on the road when it is raining.

In another example, the artificial intelligence component1002can detect a pattern that an autonomous vehicle's traction control system is less effective when the weight of the autonomous vehicle associated with passengers and cargo exceeds a certain weight. In this example, the artificial intelligence component1002can train the analysis component108to take into account the aggregate weight of passengers and cargo with regard to certain types of next actions determined or inferred by the analysis component108.

In another example, the artificial intelligence component1002can employ crowdsourcing to facilitate the determination or inference of the next action of an autonomous vehicle by the analysis component108. For example, the artificial intelligence component1002can utilize data associated with how similar autonomous vehicles operate in certain locations under various conditions and detect patterns that can facilitate the determination or inference of the next action of an autonomous vehicle by the analysis component108. In one example, the artificial intelligence component1002can determine or infer the optimal speed for an autonomous vehicle on certain curvy, mountain roads based upon data compiled from similar autonomous vehicles that have traveled such roads.

In this regard, the artificial intelligence component1002can perform classifications, correlations, inferences and/or expressions associated with principles of artificial intelligence. For instance, the artificial intelligence component1002can employ an automatic classification system and/or an automatic classification. In one example, the artificial intelligence component1002can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to learn and/or generate inferences. The artificial intelligence component1002can employ any suitable machine-learning based techniques, statistical-based techniques and/or probabilistic-based techniques. For example, the artificial intelligence component1002can employ expert systems, fuzzy logic, SVMs, Hidden Markov Models (HMMs), greedy search algorithms, rule-based systems, Bayesian models (e.g., Bayesian networks), neural networks, other non-linear training techniques, data fusion, utility-based analytical systems, systems employing Bayesian models, etc. In another aspect, the artificial intelligence component1002can perform a set of machine learning computations. For example, the artificial intelligence component1002can perform a set of clustering machine learning computations, a set of logistic regression machine learning computations, a set of decision tree machine learning computations, a set of random forest machine learning computations, a set of regression tree machine learning computations, a set of least square machine learning computations, a set of instance-based machine learning computations, a set of regression machine learning computations, a set of support vector regression machine learning computations, a set of k-means machine learning computations, a set of spectral clustering machine learning computations, a set of rule learning machine learning computations, a set of Bayesian machine learning computations, a set of deep Boltzmann machine computations, a set of deep belief network computations, and/or a set of different machine learning computations.

FIG.11illustrates a block diagram of another example, non-limiting system that facilitates displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. In certain embodiments, the system1100includes a context component1102that can monitor context of an autonomous vehicle or passengers of the autonomous vehicle. For example, the context component1102can determine preferences of one or more passengers that can be utilized by the analysis component108to determine or infer next action of the autonomous vehicle. In one example, the context component1102can determine if one or more passengers have a preference for speed over comfort. A passenger traveling to an important business meeting or a group of passengers traveling to a play can have a preference for arriving on time over the comfort of the trip. In this example, the context component1102can determine that a faster but less comfortable route that includes bumpy roads and sharp turns can be selected over a more comfortable route that will take longer, and the analysis component108can utilize this information associated with route selection to determine or infer next action.

In another example, a passenger can be more prone to motion sickness and more sensitive to bumpy roads or sharp turns. In this example, the context component1102can determine that comfort will be prioritized for route selections made for such passenger.

In another example, the context component1102can utilize data obtained by syncing with a smartphone or other external user device of one or more passengers to determine preferences of one or more passengers that can be utilized by the analysis component108to determine or infer next action of the autonomous vehicle. For example, an addition to a passenger's calendar associated with an important meeting can be utilized by the context component1102to infer that the passenger will have a preference for speed over comfort in order to be on time for the meeting.

In certain embodiments, the context component1102can identify one or more passengers of an autonomous vehicle and log data associated with each passenger. In one example, the context component1102can identify frequent passengers of an autonomous vehicle based upon profiles created by the context component1102of frequent passengers. For example, with respect to an autonomous vehicle used by a family, the context component1102can create profiles of each family member. In one example, information used by the context component1102to create profiles can be collected using questionnaires. In another example, the context component1102can utilize third party sources of information such as driver or passenger insurance data. Profiles of drivers can also be updated over time by logging a passenger's history of using the autonomous vehicle. In another example, the context component1102can identify a frequent passenger using a variety of factors. For example, the context component1102can identify a frequent passenger using cameras and facial recognition systems. In another example, the context component1102can ask or confirm the identity of a passenger using prompts in the autonomous vehicle's touch screen controls. In another example, the context component1102can identify a frequent passenger by syncing with the passenger's smartphone or other external user device. In another example, using profiles of frequent passengers, the context component1102can log data such as a passenger's preferences associated with the operation of the autonomous vehicle or routes selected by the autonomous vehicle.

FIG.12illustrates a flow diagram of an example of a method to facilitate displaying next action of an autonomous vehicle in accordance with one or more embodiments described herein. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. Box1202represents a first act that includes determination or inference of next action of an autonomous vehicle (e.g., via the analysis component108). At box1204, a graphical user interface that visually conveys the next action is generated, wherein the graphical user interface comprises a shape that dynamically morphs to visually represent the next action (e.g., via the display component110).

In certain embodiments, at box1204, the shape comprises a horizontal line to visually represent a stationary autonomous vehicle. In another embodiment, at box1204, the shape comprises a horizontal line that dynamically morphs to form a directional angle to visually represent the next action.

FIG.13illustrates another basic method flowchart1300of functional acts within various embodiments. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The method to facilitate displaying next action of an autonomous vehicle illustrated inFIG.13can be implemented in the system100ofFIG.1. As such, reference is to be made to the example ofFIG.1in the following discussion of the example ofFIG.13.

Thus, in the example ofFIG.13, a sequence to facilitate displaying next action of an autonomous vehicle1300is outlined. The sequence begins at box1302where a route is initiated based upon a selected destination and baseline determination of sequence of next actions of the autonomous vehicle. At box1304, next action is visually represented in the form of a morphable shape. At decision box1306, it is determined if a change in next action is required based upon new data. For example, the presence of pedestrians, cyclists, obstructions or other vehicles can be detected that can affect the next action of the autonomous vehicle. If no new data is detected (decision path1308), the next action at box1304is unchanged. If new data is detected (decision path1310), at box1312, next action is determined or inferred based upon the new data. At box1314, the new next action is visually represented in the form of a new morphable shape.

FIG.14illustrates another basic method flowchart1400of functional acts within various embodiments. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity. The method to facilitate displaying next action of an autonomous vehicle illustrated inFIG.14can be implemented in the system1000ofFIG.10. As such, reference is to be made to the example ofFIG.10in the following discussion of the example ofFIG.14.

Thus, in the example ofFIG.14, a sequence to facilitate displaying next action of an autonomous vehicle1400is outlined. The sequence begins at box1402where data is collected associated with the autonomous vehicle and the initial route selected. For example, collected data can include data received from systems and devices within the autonomous vehicle as well as extrinsic information (e.g., weather, traffic data . . . ). At box1404, the collected data associated with the autonomous vehicle, the initial route selected and historical information associated with operation of the autonomous vehicle is analyzed. At box decision1406, it is determined if the collected data may affect determination of next action of the autonomous vehicle. If the collected data would not affect the determination or inference of next action (decision path1408), no change in next action is determined or inferred without additional information collected1402. If the collected data would potentially affect the determination or inference of next action (decision path1410), at box1412, a utility-based analysis is performed to determine or infer next action. At box1414, one or more patterns can be detected that can facilitate determination or inference of next action.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. Computer readable program instructions for carrying out operations of the present invention can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the Figures. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

While the subject matter has been described above in the general context of computer-executable instructions of a computer program product that runs on a computer and/or computers, those skilled in the art will recognize that this disclosure also can or can be implemented in combination with other program modules. The illustrated aspects can also be practiced in distributed computing environments in which tasks are performed by remote processing devices that are linked through a communications network. However, some, if not all aspects of this disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

What has been described above include mere examples of systems and computer-implemented methods. It is, of course, not possible to describe every conceivable combination of components or computer-implemented methods for purposes of describing one or more embodiments, but one of ordinary skill in the art can recognize that many further combinations and permutations of these embodiments are possible. The descriptions of the various embodiments have been presented for purposes of illustration but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Furthermore, to the extent that the terms “includes,” “has,” “possesses,” and the like are used in the detailed description, claims, appendices and drawings such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. Moreover, articles “a” and “an” as used in the subject specification and annexed drawings should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. As used herein, the terms “example” and/or “exemplary” are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as an “example” and/or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.