Patent Publication Number: US-9889861-B2

Title: Autonomous car decision override

Description:
PRIORITY/INCORPORATION BY REFERENCE 
     This application claims priority to U.S. Provisional Application 62/324,646 entitled “Autonomous Car Decision Override” filed on Apr. 19, 2016, which is incorporated herein, in its entirety, by reference. 
    
    
     BACKGROUND 
     Currently, passengers of autonomous vehicles have no way to identify the actions the autonomous vehicle will take when a potentially hazardous obstacle, such as a pedestrian, vehicle, animal, or any other obstacle, is detected in the path of the vehicle. Furthermore, autonomous vehicles lack visual indicators to alert passengers that the vehicle is planning on changing speed, stopping, or performing another driving maneuver. This can sometimes build anxiety for the passengers because they do not know what will happen next and also they do not have a way to change the default decision made by the autonomous vehicle. In short, it would be advantageous if there was a way to indicate or convey to the passengers sitting inside the vehicle the next steps that the autonomous vehicle is going to take. 
     As autonomous vehicles evolve, there is a possibility that autonomous vehicles will not even have a steering wheel in it, so there wouldn&#39;t be an option to override autonomous vehicle&#39;s decisions. However, if the autonomous vehicle itself provides alternative decision options to the passengers keeping the safety in mind, and thereafter, if passengers were able to select those alternative decisions to override vehicle&#39;s default decisions, then that would be really helpful, however, such above mentioned feature currently does not exist. 
     A similar situation to above is that if there was no road or a big ditch in the upcoming path of the autonomous vehicle, however, in case if the autonomous vehicle&#39;s default decision was to still proceed on that path, then currently the passengers of the autonomous self driving vehicles do not have a way of knowing what actions or next steps the autonomous vehicle would take until that action is taken. At the same time, the autonomous vehicle passengers also currently do not have a way to change the default decision made by the autonomous vehicle. However, if the passengers were able to select alternative decisions to override vehicle&#39;s default decisions, then that would be really helpful, however, such above mentioned features currently do not exist in the autonomous self-driving vehicles. 
     According to already existing patents granted, autonomous vehicles will indicate to the outside pedestrians about the actions that the autonomous vehicle is about to take. However, there is currently no feature within the autonomous vehicles to display the next steps or actions that the vehicle would take to the passengers sitting inside the vehicle. Also, there is currently no option for passengers to override the autonomous vehicle&#39;s default decisions by selecting the alternative decision options that are presented by the autonomous vehicle itself by keeping the safety in mind. 
     SUMMARY 
     Described is a visual display for an autonomous vehicle. The visual display includes a first display section that displays an external view of the autonomous vehicle, a second display section that displays vehicle actions that the autonomous vehicle will take and a third display section that displays alternative vehicle actions that an authorized passenger of the autonomous vehicle may select to override the vehicle actions. 
     Further described is a method for controlling an autonomous vehicle. The method includes displaying an external view of the autonomous vehicle, displaying vehicle actions that the autonomous vehicle will take, displaying alternative vehicle actions that the autonomous vehicle may take, receiving a selection of the alternative vehicle actions from an authorized passenger of the autonomous vehicle and overriding the vehicle actions with the selected alternative vehicle actions. 
     In addition, a further visual display for an autonomous vehicle is described. The further visual display includes a first display section that displays vehicle actions that the autonomous vehicle will take and a second display section that displays alternative vehicle actions that an authorized passenger of the autonomous vehicle may select to override the vehicle actions. 
     A decision system for an autonomous vehicle, comprising an input sensor to collect information from passengers to determine if a passenger is an authorized passenger and a display section that displays alternative vehicle actions that an authorized passenger of the autonomous vehicle may select to override vehicle actions of the autonomous vehicle. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  shows a visual display that is used to inform passengers of the decisions made by an autonomous vehicle according to the exemplary embodiments. 
         FIG. 2  shows an exemplary alternative mechanism to select alternative decision options via a vehicle steering wheel with a button according to the exemplary embodiments. 
         FIG. 3  shows an alternative exemplary embodiment to be able to select the alternative decision options listed on the display of  FIG. 1 . 
     
    
    
     DETAILED DESCRIPTION 
     The present invention may be further understood with reference to the following description and the appended drawings, wherein like elements are provided with the same reference numerals. The exemplary embodiments show a system, method and apparatus to inform passengers of the decisions made by an autonomous vehicle, based on external conditions, and provide the passenger with the ability to select alternative decisions, thus commanding the autonomous vehicle while the vehicle remains in autonomous driving mode. 
     It should be noted that the terms “decisions,” “vehicle decisions,” “actions,” and “vehicle actions” may be used interchangeably throughout this description. In this description, these terms are used to describe the actions that will be taken by the autonomous vehicle based on the vehicle&#39;s programming. For example, based on the external factors that the autonomous vehicle senses, the vehicle&#39;s software may decide to slow down, increase speed, maintain the current speed, come to a stop, alter the course of the vehicle, make a turn, etc. and cause the autonomous vehicle to perform the selected action. In addition, the term “alternative” may be used to modify these above terms. Where the term “alternative” is used to modify these terms, it indicates that an authorized passenger may select a different vehicle action or decision that is different from the one selected by the vehicle&#39;s programming. 
       FIG. 1  shows a visual display  10  that is used to inform passengers of the decisions made by an autonomous vehicle, based on external conditions, and provide the passenger with the ability to select alternative decisions, thus commanding the autonomous vehicle while the vehicle remains in autonomous driving mode. The display  10  may be of any type including but not limited to LCD, LED, or haptic displays. The display  10  may be located in any part of the vehicle, where it can either be in the front, left, right, top, bottom, or center of the vehicle. The display  10  may be either at a single or multiple locations in the autonomous vehicle. In one example, there may be a single display  10  that is within the reach of the person in the driver&#39;s seat. In another example, the single display  10  may be mounted in a center console such that the persons in the front seats may access the display  10 . In a further example, each seat of the autonomous vehicle may have access to a display  10 , e.g., the rear of the front seats may each include a display  10  that are accessible to the persons in the back seats. Also, the display  10  may be affixed, attached, or permanently mounted in the autonomous vehicle. 
     In the exemplary embodiment of  FIG. 1 , the display  10  is a touchscreen and comprises section  13  that continuously displays the external view where the vehicle is going along with the potentially hazardous external obstacles such as a pedestrian, vehicle, animal, ditch, or any other obstacle identified by sensors and/or computer processing. The display  10  may identify the potentially hazardous external obstacles. The identification of the potentially hazardous external obstacles may be via a graphical display on the display  10 , a highlight on the display  10 , an outline on the display  10  or any other manner of indicating to the passengers of the vehicle that there is a potentially hazardous external obstacle. The external view as shown in section  13  may comprise a front view, rear view, side views, or any other view depending upon the vehicle&#39;s movement. The external view of section  13  may also comprise the overlaid simulated navigation path of the autonomous vehicle to display the upcoming decisions and proceeding actions the autonomous vehicle plans to take based on sensor input from the external environment. This simulated navigation path of the autonomous vehicle can be demonstrated in a number of ways comprising either independent or a combination of lines, arrows, images, moving images, animation, symbols, or any other simulated navigation path. 
     The section  13  may also display any sensor malfunctions of the autonomous vehicle or if a sensor is not functioning, informing the passenger of the vehicle&#39;s diminished capacity to make informed decisions. 
     The display  10  of  FIG. 1  also comprises a section  11  listing the decisions and proceeding actions the vehicle will take based on sensor input from the external environment. In addition to listing the next steps that the vehicle is about to take, section  11  may also comprise additional visual indicators that indicate to the passengers if the vehicle is about to slow down, increase speed, maintain the current speed, come to a stop, make a turn, etc. The section  11  may also display any sensor malfunctions of the autonomous vehicle or if a sensor is not functioning, informing the passenger of the vehicle&#39;s diminished capacity to make informed decisions. 
     In addition to listing the decisions and proceeding or next actions the vehicle will take in section  11  of display  10 , there may also be visual indicators located in different parts of the vehicle that indicate to the passengers if the vehicle is about to slow down, increase speed, maintain the current speed, come to a stop, make a turn, etc. Such visual indicators may be affixed, attached or permanently mounted in the vehicle at any location, wherein even if the passengers were not constantly looking at display  10 , they would still be able to quickly look at the visual indicators located in one or multiple locations around the vehicle and know the next steps that the autonomous vehicle is about to take. 
     The display  10  of  FIG. 1  also comprises a section  12  that provides for alternative decision options that can be selected by an authorized passenger if they want to override the default decisions listed in section  11 . The alternative decision options may comprise altering vehicle course, stopping the vehicle, slowing down the vehicle, increasing the speed of the vehicle, etc. 
     An authorized passenger may be anyone designated by the vehicle owner as a person approved to override decisions made by an autonomous vehicle. 
     An alternative to continuous display of potentially hazardous external obstacles on section  13  of display  10  may be an inquiry function within section  13  itself or an external independent function that allows a passenger to inquire if the vehicle has detected a potentially hazardous external obstacle such as a pedestrian, vehicle, animal, ditch, etc. When the inquiry function is selected, section  13  will display the external obstacles the vehicle has detected, section  11  will display the default decisions made by the autonomous vehicle based on one or more external obstacles identified, and section  12  will provide alternative decision options for the passenger comprising of altering vehicle course, stopping the vehicle, slowing down the vehicle, increasing the speed of the vehicle, etc. 
     A substitute for section  12  of  FIG. 1  that displays a list of alternative decision options may be that the section  13  itself displays the alternative decision options overlaid directly on the external view of section  13 , wherein an authorized passenger would be able to select one or more of those alternative decision options and override the default decision of the autonomous vehicle. These alternative decision options may be integrated or overlaid on the external view of section  13  in a number of ways comprising of either independent or a combination of lines, arrows, images, moving images, animation, symbols, different color, etc. and would not be limited to just the ways mentioned above as long as those alternative decision options can be overlaid on section  13 . 
     Another substitute for section  12  of  FIG. 1  that displays a list of alternative decision options may be that the section  11  displays the alternative decision options in a new or a pop-up window when an authorized passenger selects an existing default decision or one of its subsets in section  11 . In this case, an authorized passenger would be able to select one or more of those alternative decision options and override the default decision of the autonomous self-driving vehicle using section  11 . 
     In addition, an alternative to section  11  of  FIG. 1  that displays default decisions made by an autonomous vehicle, may be to overlay those default decisions on the external view of section  13  itself. In this case, passengers would be able to view the default decisions made by the autonomous vehicle directly on section  13 . 
       FIG. 1  also shows a knob  14  in close proximity to display  10 . The knob  14  provides the authorized passenger with the ability to select alternative decision options displayed in section  12  if display  10  is a non-touch type screen or as an alternative to using a touch sensitive display  10 . In case if the alternative decision options are displayed in section  11  or section  13 , then the knob  14  would still provide the authorized passenger with the ability to select alternative decision options in those sections. 
       FIG. 2  also shows an alternative mechanism to select alternative decision options via a vehicle steering wheel with a button  21  that provides for voice operated functionality and allows the authorized passenger in the autonomous vehicle to select an alternative decision from section  12  of  FIG. 1  via the passenger&#39;s voice, as received from various microphone sensors throughout the vehicle. The autonomous vehicle recognizes only the voice of authorized passengers. The authorized passenger&#39;s voice is recognized by the autonomous vehicle and the vehicle reacts accordingly. The passenger may select alternative decision options from section  12  in  FIG. 1  via a voice command by holding button  21 . It should be noted that, as described above, the voice input may be triggered by a button  21 . However, there may be other manners of triggering the voice input. For example, the voice input may simply be triggered by an authorized passenger starting to speak and the voice input recognizing the authorized passenger&#39;s voice. In another example, the passenger may say a keyword to activate the decision system. The keyword may be, for example, the name of the system, a keyword programmed by the passenger, a keyword programmed by the vehicle maker, etc. 
     In case the alternative decision options are displayed in either section  11  or section  13  of  FIG. 1  rather than section  12  of  FIG. 1 , then also holding the button  21  of  FIG. 2  would allow the authorized passengers in the autonomous vehicle to be able to select those alternative decision options using the voice command. One or ordinary skill in the art would recognize that in order to command the vehicle, the authorized passengers of an autonomous vehicle can initiate their voice commands in many different ways without departing from the spirit and scope of the present invention and therefore would not be limited to any of the examples given above. 
       FIG. 2  also shows an example for adding an authorized passenger so that the vehicle will recognize the passenger&#39;s voice, a fingerprint scanning sensor  22 . An owner of the vehicle would be asked to set up their authorization using the fingerprint scanning sensor  22  when they first purchase the autonomous vehicle. For example: by using scanning sensor  22 , an owner would add their fingerprint to the vehicle&#39;s onboard processing unit, making them an authorized passenger. If the owner wants, then they would also be able to add additional authorized passengers and allow those passengers to also add their fingerprints to the onboard processing unit. Any of the authorized passengers of the autonomous vehicle would be able to pre-record their voice so the autonomous vehicle recognizes their voice later when the voice control is activated. It should be noted that any biometric sensor may be substituted for the fingerprint scanning sensor  22 , e.g., a handprint sensor, a retina scan sensor, etc. 
     One or ordinary skill in the art would recognize that a fingerprint scanning sensor  22  can be placed any part of the autonomous vehicle and can be located at either single or multiple locations without departing from the spirit and scope of the present invention. 
     Authorized passengers would be able to use their voice to command the vehicle such as select alternative decision options, inquire if the vehicle has detected potentially hazardous external obstacles, etc. In other words, once a fingerprint has been stored, the authorized passenger may command the vehicle via voice button  21  and verify their identity as an authorized passenger via scanning sensor  22  or through voice recognition. Alternatively, an authorized passenger may also select alternative decision options and operate display  10  via the directional pad button  23 . The button  23  provides for full operation of display  10  of  FIG. 1  and allows the authorized passenger to select alternative decision options in section  12  of  FIG. 1  to command the vehicle. 
       FIG. 3  shows an alternative exemplary embodiment to be able to select the alternative decision options listed in the section  12  on display  10  of  FIG. 1 . Autonomous vehicles do not require a steering wheel, and as such voice activation buttons can no longer be centrally located on the steering wheel in vehicles without a steering wheel.  FIG. 3  shows a dashboard  30  with a centrally located display  10  and various voice control buttons  31 . The voice control button  31  provides the same functionality of button  21  in  FIG. 2 . Pressing button  31  would allow the authorized passenger in the autonomous vehicle to select an alternative decision from section  12  of  FIG. 1  via the passenger&#39;s voice, as received from various microphone sensors throughout the vehicle. The autonomous vehicle recognizes only the voice of authorized passengers. The authorized passenger&#39;s voice is recognized by the autonomous vehicle and the vehicle reacts accordingly. The passenger may select alternative decision options from section  12  in  FIG. 1  via a voice command by holding button  31 . 
     An alternative to authorizing a passenger of an autonomous vehicle instead of a fingerprint scanner can be a facial recognition, optical/retinal recognition, ear shape recognition, other biometric recognition, or verbal or hand-entered passcode verification. Additionally, a passenger who enters a passcode or inserts the vehicle key would be allowed to view the alternate decision options as shown in section  12  of  FIG. 1  on their personal digital screen, which would be within reach of their seat in the autonomous vehicle. Thus, any type of input sensor may be used to verify an authorized passenger. One of ordinary skill in the art would recognize that authorizing passengers of an autonomous vehicle can be done in many different ways as long as that authorization allows the passenger to command the autonomous vehicle. Therefore, any of such authorization methods would be within the spirit and scope of the present invention and would not be limited to the examples provided above. 
     An alternative to requiring an autonomous vehicle&#39;s passenger to be authorized before they can override the vehicle&#39;s default decisions can be that a particular seat in the vehicle can be assigned as the authorized seat, wherein the passenger that sits on that authorized seat would have access to review, inquire or especially override the autonomous vehicle&#39;s default decision. Similar to a driver seat in the traditional non-autonomous vehicles, there can be an authorized driver seat in the autonomous vehicle, wherein such passenger sitting on that authorized seat would automatically have the options available to be able to override the autonomous vehicle&#39;s default decisions. In this case, there would be no special requirement to authorize the passenger using mechanisms like a fingerprint scanner, facial recognition, password, or other authorization method before the passenger could review, inquire or override the autonomous vehicle&#39;s default decision. 
     An alternative to requiring an autonomous vehicle&#39;s passenger to be authorized before they can override the vehicle&#39;s default decisions can be that any of the passengers sitting in the autonomous vehicle would be allowed to override the vehicle&#39;s default decisions i.e. any passenger could select alternative decision options so they can override the vehicle&#39;s default decisions. In this case, there would be no special requirement to authorize the passenger using mechanisms like fingerprint scanner, facial recognition, password, or other authorization method before the passengers could review, inquire or override the autonomous vehicle&#39;s default decision. 
     The exemplary embodiments may also comprise an artificial intelligence module, wherein the vehicle would learn the common alternative decision options that the authorized vehicle passenger usually selects under different manual intervention situations, and based on that artificial intelligence knowledge, system would quickly present those commonly selected alternative decision options to the users for future similar situations. Thus, the system would continue to learn and help the vehicle passengers with the future selection of their alternative decision options. 
     The above mentioned artificial intelligence module would also learn from the repeated actions of the authorized passenger based on different situations and accordingly alter the default decisions of the autonomous vehicle, so in future similar situations, an authorized passenger wouldn&#39;t have to do a manual intervention each time by selecting an alternate decision option. An example of this learning would be: if the authorized passenger, after seeing a child on the road, always selects the alternative decision option to stop the vehicle at a distance, then the artificial intelligence module of the autonomous vehicle would learn such behavior and accordingly alter the default decisions of the autonomous vehicle for similar future situations to meet the passenger&#39;s preference. 
     Any of the learning captured by artificial intelligence module would still consider safety of the passengers and the pedestrians as the highest priority and would not present an alternative decision option to the passengers that could impact or jeopardize safety. In section  12  of  FIG. 1 , any of the alternative decision options provided would always be such that they consider safety of the passengers and the pedestrians as the highest priority and would not present an alternative decision option to the passengers that could impact or jeopardize safety. 
     It will be appreciated by those skilled in the art that various modifications and alterations of the exemplary embodiments can be made without departing from the scope of the appended claims. Some of these have been discussed above and others will be apparent to those skilled in the art.