Enhanced touchscreen operation

A location of a first wearable device is determined based on a first biometric signature. Operation of a touchscreen display is restricted if the location of the first wearable device is in an operator's side of a vehicle cabin and the first wearable device is within a first distance from the display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage of, and claims priority to, Patent Cooperation Treaty Application No. PCT/US2015/058578, filed on Nov. 2, 2015, which application is hereby incorporated herein by reference in its entirety.

BACKGROUND

Touchscreen displays allow vehicle occupants to interact with vehicle systems. However, to avoid distracting a vehicle operator, e.g., driver while a vehicle is in operation, touchscreen displays may deactivate completely during vehicle operation, preventing vehicle passengers from using the touchscreen display.

DETAILED DESCRIPTION

FIG. 1illustrates a system100including a wearable device140communicatively coupled to a vehicle101computing device105. The computing device105is programmed to receive collected data115from one or more data collectors110, e.g., vehicle101sensors, concerning various metrics related to the vehicle101. For example, the metrics may include a velocity of the vehicle101, vehicle101acceleration and/or deceleration, data related to vehicle101path or steering including lateral acceleration, curvature of the road, biometric data related to a vehicle101operator, e.g., heart rate, respiration, pupil dilation, body temperature, state of consciousness, etc. Further examples of such metrics may include measurements of vehicle systems and/or components (e.g. a steering system, a powertrain system, a brake system, internal sensing, external sensing, etc.).

The computing device105is generally programmed for communications on a controller area network (CAN) bus or the like. The computing device105may also have a connection to an onboard diagnostics connector (OBD-II). Via the CAN bus, OBD-II, and/or other wired or wireless mechanisms, the computing device105may transmit messages to various devices in a vehicle and/or receive messages from the various devices, e.g., controllers, actuators, sensors, etc., including data collectors110. Alternatively or additionally, in cases where the computing device105actually comprises multiple devices, the CAN bus or the like may be used for communications between devices represented as the computing device105in this disclosure. In addition, the computing device105may be programmed for communicating with a network120, which, as described below, may include various wired and/or wireless networking technologies, e.g., cellular, Bluetooth, wired and/or wireless packet networks, etc.

The data store106may be of any known type, e.g., hard disk drives, solid-state drives, servers, or any volatile or non-volatile media. The data store106may store the collected data115sent from the data collectors110.

The vehicle101may include a touchscreen display107. The display107may allow an operator of the vehicle101to interface with the computing device105, with electronic control units, vehicle101systems, etc. The display107may include interactive voice response (IVR) and/or a graphical user interface (GUI). The display107may present information relayed from the network120, e.g. from a server125, that extends outside of the vehicle101and may communicate directly with the computing device105, e.g., using Bluetooth, etc.

Data collectors110may include a variety of devices. For example, various controllers in a vehicle may operate as data collectors110to provide data115via the CAN bus, e.g., data115relating to vehicle speed, acceleration, system and/or component functionality, etc., of any number of vehicles101, including the host vehicle and/or the target vehicle. Further, sensors or the like, global positioning system (GPS) equipment, etc., could be included in a vehicle and configured as data collectors110to provide data directly to the computer105, e.g., via a wired or wireless connection. Sensor data collectors110could include mechanisms such as RADAR, LIDAR, sonar, etc. sensors that could be deployed to measure a distance between the vehicle101and other vehicles or objects. Yet other data collectors110could include cameras, breathalyzers, motion detectors, biometric sensors that can detect, e.g., a heartbeat, etc., i.e., data collectors110to provide data115for evaluating a condition or state of a vehicle101operator.

Collected data115may include a variety of data collected in a vehicle101. Examples of collected data115are provided above, and moreover, data115is generally collected using one or more data collectors110, and may additionally include data calculated therefrom in the computing device105, and/or at the server125. In general, collected data115may include any data that may be gathered by the data collectors110and/or computed from such data. The collected data115may be presented on the display107.

The system100may further include the network120connected to the server125and a data store130. The computer105may further be programmed to communicate with one or more remote sites such as the server125, via the network120, such remote site possibly including a data store130. The network120represents one or more mechanisms by which a vehicle computer105may communicate with a remote server125. Accordingly, the network120may be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber) and/or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when multiple communication mechanisms are utilized). Exemplary communication networks include wireless communication networks (e.g., using Bluetooth, IEEE 802.11, etc.), local area networks (LAN) and/or wide area networks (WAN), including the Internet, providing data communication services.

The server125may be programmed to determine an appropriate action for one or more vehicles101, and to provide direction to the computer105to proceed accordingly. The server125may be one or more computer servers, each generally including at least one processor and at least one memory, the memory storing instructions executable by the processor, including instructions for carrying out various steps and processes described herein. The server125may include or be communicatively coupled to a data store130for storing collected data115, records relating to potential incidents generated as described herein, biometric information on vehicle101occupants, etc. Further, the server125may store information related to particular vehicle101and additionally one or more other vehicles101operating in a geographic area, traffic conditions, weather conditions, etc., within a geographic area, with respect to a particular road, city, etc. The server125could be programmed to provide alerts and/or messages to a particular vehicle101and/or other vehicles101. The server125could be programmed to present information to a touchscreen display107.

A wearable device140may be any one of a variety of computing devices including a processor and a memory, as well as communication capabilities that is programmed to be worn on an operator's body. For example, the wearable device140may be a watch, a smart watch, a vibrating apparatus, etc. that includes capabilities for wireless communications using IEEE 802.11, Bluetooth, and/or cellular communications protocols. Further, the wearable device140may use such communications capabilities to communicate via the network120and also directly with a vehicle computer105, e.g., using Bluetooth.

The wearable device140may include a biometric sensor145to collect data about a vehicle101occupant. For example, the biometric sensor may measure a heartbeat of a vehicle101occupant and/or other cardiac data, e.g., a variation in heart rate, a blood pressure, or a QT interval. The QT interval, as is known, is the measure of time between the start of the Q wave and the end of the T wave in the heart's electrical cycle. The QT interval represents electrical depolarization and repolarization of the heart's ventricles, and is a known cardiac measurement that can be distinguished between several occupants. The biometric sensor145may collect other biometric data, e.g., a skin galvanic response as is known.

FIG. 2illustrates a process200for determining whether a wearable device140belongs to a vehicle101operator, e.g., a driver, or a vehicle101passenger. The process200starts in a block205, in which the computing device105determines whether the biometric sensor145detects a biometric signature. The computing device105may use biometric data collected by the biometric sensor145, e.g., the QT interval, a pattern of variation in heart rate, electrocardiogram readings, blood oxygen content, blood pressure (systolic and diastolic), galvanic skin response, etc., from data indicated by the biometric sensor145to determine the biometric signature. The biometric signature indicates that a vehicle101occupant is wearing the wearable device140that includes the biometric sensor145. If the biometric sensor145does not detect a biometric signature, the computer105determines that the wearable device140is not being worn by the vehicle101occupant. The computing device105may alternatively detect the biometric signature via a biometric sensor110in a vehicle101steering wheel. If the biometric sensor145(or a data collector110) detects a biometric signature, the process200continues in a block210. Otherwise, the process200ends.

In the block210, the computing device identifies the wearable device140associated with the biometric signature detected in the block205, i.e., the wearable device140that includes the biometric sensor145that detected the biometric signature.

Next, in a block215, the computing device105links the biometric signature to the first wearable device140, i.e., stores an association of the detected biometric signature with the first device140identified in the block210. Biometric data differ between vehicle101occupants, and the biometric signature may be a single type of biometric data or a collection of some or all of the biometric data for a particular occupant. The computing device105may link the first wearable device140with the biometric signature.

Next, in a block220, the computing device105determines whether the first wearable device140is near, i.e., within a predetermined threshold distance of, a vehicle101steering wheel. The computing device105may collect data115from one of the data collectors110, e.g., Bluetooth proximity sensing, to determine whether the first wearable device140is within the threshold distance from the steering wheel. The threshold distance is established to allow a determination of whether the first wearable device140is close enough to the steering wheel such that the first wearable device is being worn by the operator of the vehicle101. An exemplary threshold distance could be 80-120 mm. If the first wearable device is within the threshold distance from the steering wheel, the process200continues in a block225. Otherwise, the process200continues in a block230.

For example, as shown inFIG. 4, the vehicle101may have a plurality of detection zones108. The detection zones108may include data collectors110, e.g. Bluetooth proximity sensors configured to detect a received signal strength indicator, being a measure of the power present in a signal, e.g. a Bluetooth signal, to detect whether a wearable device140is within one of the detection zones108. The detection zones108may be located around, e.g., a vehicle101steering wheel, the touchscreen display107, and/or a vehicle seat. The detection zones108may be determined at least in part by the shape of the interior of the vehicle101and the predicted location(s) of the occupants and their wearable devices140. For example, inFIG. 4, one of the detection zones108may form an annulus around the vehicle101steering wheel, e.g., within a range of 80-120 mm from the steering wheel. Another detection zone108may form an annulus around the touchscreen display107, e.g., 290-410 mm from the display107. Another detection zone108may encompass the front passenger seat, e.g., extending laterally from a passenger door armrest to a center armrest, longitudinally from a seat back to the front of a seat cushion, and vertically from the cushion to the top of a headrest. The detection zones108may be modified based on placement of the data collectors110. The computing device105may use the detection zones108to determine whether the first wearable device140is within the threshold distance of the steering wheel and/or the touchscreen display107.

In the block225, the computing device105designates the wearable device140as the operator's wearable device140. Thus, if the operator's wearable device140approaches the touchscreen display107, then the operator is likely attempting to use the touchscreen display107. The process then continues in a block235.

In the block230, the computing device105designates the wearable device140, determined to have been at or greater than the threshold distance from the steering wheel, as a passenger's wearable device140. Thus, if this designated wearable device140approaches the touchscreen display107, the passenger, rather than the vehicle101operator, e.g., driver, is likely attempting to use the touchscreen display107. In another example, the computing device105may further provide an instruction to a data collector110in a passenger's seat, e.g. an occupant classification sensor, to measure a weight present in the passenger's seat. The computing device105may designate the wearable device140a passenger's wearable device140only if the computing device105confirms that there is a weight comparable to an occupant in the passenger's seat, i.e. a passenger is present in the passenger's seat. The process then continues in the block235.

In the block235, which may follow either of the blocks225,230, the computing device105determines whether a second heartbeat is detected. That is, the computing device105searches for a second wearable device140with a second biometric sensor145detecting a second biometric signature from a second vehicle101occupant. For example, the computing device105may receive biometric information and determine that the signature is a second biometric signature if the biometric data differ from the first biometric data for a predetermined period, e.g. a heartbeat differs for a number of seconds, a number of heartbeats, etc. If the computing device105detects a second biometric signature, the process200returns to the block210to detect a second wearable device, or as many wearable devices and occupants as necessary. Otherwise, the process200ends.

FIG. 3illustrates a process300for allowing use of the touchscreen display107based on proximity of wearable devices140to the display107. The process300may follow or be combined with the process200, e.g., may use the determination of whether one or more wearable devices detected as described with respect to the process200are associated with a vehicle101operator or passenger.

The process300starts in a block305, in which the computing device105determines whether the operator's wearable device140is within a first threshold distance from the touchscreen display107. The touchscreen display107may include a data collector110, e.g. a Bluetooth receiving element, that detects the distance between the touchscreen display107and the wearable device140. For example, the first threshold distance may indicate a distance from the wearable device140to the touchscreen display107that indicates that the operator is attempting to use the touchscreen display. The first threshold distance may be, e.g., 155-200 millimeters (mm). The distance between the operator's wearable device140and the touchscreen display107during operation of the vehicle101may be, e.g., 290-410 mm. The computing device105may alternatively determine whether the operator's wearable device140is within the detection zone108around the touchscreen display107. If the operator's wearable device140is within the first threshold distance, the process300continues in the block310. Otherwise, the process continues in a block315.

In the block310, the computing device105restricts operation of the touchscreen display107. If the operator's wearable device140is within the first threshold distance, the operator may attempt to use the touchscreen display107. Restricting operation the touchscreen display107prevents the operator from using certain functions of the touchscreen display107and being distracted during operation of the vehicle101. The touchscreen display107may be restricted, i.e., certain functions of the touchscreen display107may be rendered inoperable, e.g. a navigation system, and/or a number of keystrokes within a given function may be limited. Alternatively, the touchscreen display107may be disabled entirely, i.e., the touchscreen display107will not accept any input from an occupant touching it. In another example, the display107is disabled by default, and if the computing device105does not detect both an operator's wearable device140and a passenger's wearable device140in the process200, the computing device105detects only one biometric signature, or the computing device105does not detect a weight in the passenger's seat from the occupant classification sensor, the display107will remain disabled. The process300then continues in a block325.

In the block315, the computing device105determines whether the passenger's wearable device140is within the first threshold distance of the display107. For example, the computing device105may determine whether the passenger's wearable device140is within the detection zone108of the display107. The operator may wear the operator's wearable device140on a wrist opposite a hand closest to the display107. Thus, the operator's wearable device may be beyond the first threshold distance but the hand without the operator's wearable device140may attempt to use the display107. To prevent the use of the display107with the operator's free hand, the computing device105may be programmed to activate the touchscreen display107when the operator's wearable device140is beyond the first threshold distance and the passenger's wearable device140is within the first threshold distance. If the passenger's wearable device140is within the first threshold distance, the process300continues in a block317. Otherwise, the process300continues in the block310.

In the block317, the computing device105confirms that the operator's wearable device140and the passenger's wearable device140detect different biometric signatures. If the passenger removes the passenger wearable device140and the operator wears the passenger wearable device140, and the process200does not determine that both wearable devices140are worn by the operator, the computing device105may still restrict operation of the touchscreen107. The computing device105may compare the biometric signature detected from the passenger's wearable device140to the biometric signature detected from the operator's wearable device140, and the biometric signatures of the two devices140should not match for at least one of a software configurable value of consecutive heart beats, e.g. 4 beats, and/or for a software configurable duration, e.g. 4 seconds, during unrestricted screen use. If the computing device105confirms that the wearable devices140detect different biometric signatures, the process300continues in a block320. Otherwise, the process300continues in the block310.

In the block320, the computing device105activates the touchscreen display107, i.e., removes the restrictions and/or completely enables the touchscreen display107. If the operator's wearable device140is away from, i.e., greater than a predetermined threshold distance from, the touchscreen display107, then the operator is not going to use the touchscreen display107, and the passengers may freely use the touchscreen display107as long as the passengers use a hand that brings their respective passenger's wearable device140within the first threshold distance. In the present example, where the touchscreen display107is disabled by default, the process300leading to the block320is the only manner to activate the touchscreen display107, i.e., only if the computing device105detects two wearable devices140, one worn by the operator and the other worn by the passenger, the operator's device140is within the threshold distance of the steering wheel, the passenger's device140is within the threshold distance of the touchscreen display107, and the computing device105detects two biometric signatures. The process300in this example thus prevents the operator from circumventing the disabling of the touchscreen display107by wearing both devices140after linking one of the devices140with the passenger.

In the block325, the computing device105decides whether to continue the process300. If so, the process300returns to the block305to determine whether the operator's wearable device is near the touchscreen display107. Otherwise, the process300ends.

With regard to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. For example, in the process200, one or more of the steps could be omitted, or the steps could be executed in a different order than shown inFIG. 2. In other words, the descriptions of systems and/or processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the disclosed subject matter.