Gear guard camera and bluetooth sensor fusion to enhance user experience

Systems and methods are presented herein for improving the response of a vehicle to at least one person or object approaching a vehicle, and more particularly, to a vehicle that uses at least one of sensor data or Bluetooth sensor data to determine a vehicle state and corresponding vehicle responses to at least one person or object approaching a vehicle. A signal strength is of a wireless signal received from a user device is determined. An object is detected based on a sensor configured to collect data corresponding to an environment external to a vehicle. A vehicle state is determined based on the signal strength and the detected object.

INTRODUCTION

The disclosure is generally directed to improving the response of a vehicle to at least one person or object approaching or leaving a vehicle.

SUMMARY

A vehicle may have a guard mode (e.g., a vehicle mode where a vehicle or items stored in or on the vehicle may be monitored in order to determine whether a responsive or defensive action is to be performed by various vehicle systems, such as locking doors and recording events). The guard mode may lock various doors and containers in response to determining a user has left the internal compartments of the vehicle and has left the area of the vehicle. Additionally, the guard mode may enable or activate cameras and other sensors to characterize an environment surrounding the vehicle to determine if additional locks, notifications, or alarms should be activated to alert the user to an unwelcome vehicle intruder.

The guard mode may have varying levels of activity and processing depending on how far from the vehicle the user is, while also considering whether the user is leaving or approaching the vehicle. A reduction in memory used or processing power required for each iteration of an active guard mode is advantageous in order to allow seamless ingress and egress of the user into and out of the vehicle without the user being overloaded with notifications or alerts for events that do not put the user, the vehicle, or the contents of the vehicle at risk.

Additionally, with lock and unlock functionality, one or more Bluetooth signal receivers may be used and receive wireless signals from a user device. However, because of the different locations of the Bluetooth signal receivers and the relative positions of the user device and obstructions, inconsistent signals may be received which may result in unexpected, invalid, or overall incapable response to a user's proximity to a vehicle (e.g., the doors get locked when the user approaches instead of being unlocked). This may lead to user frustration. Accordingly, improved and consistent vehicle responses to user behavior would be advantageous.

In some example embodiments, the disclosure is directed to at least one of a system configured to perform a method, a non-transitory computer readable medium (e.g., a software or software related application) which causes a system to perform a method, and a method for modifying a vehicle state based on a wireless signal from a user device and object detection data. A signal strength of a wireless signal, received from a user device, is determined. Additionally, an object is detected based on a sensor configured to collect data corresponding to an environment external to a vehicle. A vehicle state is determined based on the signal strength and the detected object. In some embodiments, the wireless signal corresponds to user identification data of a user associated with the user device and the user is external to the vehicle. The sensor is at least one of a camera, a lidar sensor, a radar sensor, a thermal sensor, an ultrasonic sensor, or any sensor suitable for capturing data corresponding to one or more of a user device and an environment surrounding the vehicle. The sensor is arranged to characterize the environment external to the vehicle.

In some embodiments, a current vehicle state has vehicle door lock status that has locked vehicle doors or unlocked vehicle doors. For example, the current vehicle state may be activated based on the activation of the guard mode which monitors an environment around the vehicle and locks the vehicle doors. Additionally, a user state may be determined which indicates that the user is external to the vehicle and is either approaching or departing the vehicle. A comparison between the vehicle state and the user state is performed. In response to determining the current vehicle state is inconsistent with a user state corresponding to a user location relative to the vehicle, modifying the current vehicle state to have a different vehicle door lock status (e.g., if the user is approaching, locked doors are unlocked or if the user is departing, unlocked doors are locked).

In some embodiments, a current vehicle state activates the sensor for collecting and processing data related to the environment surrounding the vehicle. For example, activation of the guard mode may result in power from the vehicle being provided to one or more sensors (e.g., proximity sensors and cameras) which collect data for processing circuitry to identify objects around the vehicle as well as provide input for notifications to be generated by the processing circuitry for transmission to the user device. If the processing circuitry determines that a user is approaching the vehicle (e.g., based on one or more of objection detection as shown inFIG.9or RF signal analysis as described in reference toFIG.7) and there are no other objects in the environment surrounding the vehicle, the vehicle state (e.g., guard mode) is modified such that there is a limitation on the processing of data collected and, by extent, there is a limitation on the generation of notifications for the user. As a result, the guard mode is modified to be in a reduced processing state such that additional power consumption and data processing is prevented. In some embodiments, modifying the current vehicle state comprises at least one of preventing generation of notifications that would otherwise be generated based on data collected from the sensor, preventing processing of data collected from the sensor (e.g., deactivating a guard function that operates on data collected from the sensor), or modifying the current vehicle state comprises focusing processing of data to data corresponding to visual indicators of the object (e.g., the object is the user and modifying the current vehicle state comprises continuing to monitor the user based on data collected by the sensor).

In some embodiments, determining the signal strength of the wireless signal received from the user device comprises determining a first signal parameter of the wireless signal at a first time, determining a second signal parameter of the wireless signal at a second time, and comparing the first signal parameter to the second signal parameter. In response to determining the first signal parameter is different than the second signal parameter based on the comparing, the vehicle state is modified based on a difference between the first signal parameter and the second signal parameter.

The approaches discussed are intended to address the aforementioned deficiencies by means exemplified in the following use cases. A first use case involves a user remaining within a close proximity of the vehicle (e.g., within 10 feet of the vehicle). Around the vehicle, there may be blind spots (e.g., due to obstructions) for cameras and other sensors. There may also be blind spots that result in an attenuation or blocking of radio frequency (hereinafter “RF”) signals which may be generated by at least one of a vehicle or a user device of the user which may be paired with the vehicle for user authentication. By using a sensor (e.g., a camera) along with the wireless signal sensor for the RF signals, two sets of data can be analyzed by the system to determine if a user is within a proximity of the vehicle as well as a context of the user's proximity (e.g., the user is approaching, leaving, or remaining around the vehicle, which may reduce the need for data collection and processing by the vehicle).

A second use case involves fluctuations of the RF signals which may result in a false increase of the signal strength (e.g., a user takes a phone out of their pocket while walking away from the vehicle), resulting in an interpretation that a user is approaching the vehicle. As a result, the user may be far away from the vehicle when the vehicle system determines to unlock the car. By combining RF signal data analysis with other sensor data, an improved context of the user location relative to the vehicle can be determined and a vehicle state controller can modify the vehicle state appropriately, according to a more accurate depiction of the user proximity relative to the vehicle.

A third use case involves a guard mode of a vehicle remaining active while a user remains within a small distance of the vehicle (e.g., within 5 feet or 10 feet). For example, a user may park at an event and may wander to neighboring parked cars without losing sight of their own vehicle. The guard mode (e.g., a gear guard mode) may activate cameras and sensors to capture information about the surrounding environment of the vehicle, despite the user being within a reasonable distance of the vehicle. The user may then receive excessive notifications or data about their vehicle, even though the user is able to visibly see and determine whether their vehicle (or the contents of the vehicle) are at risk. The systems and methods of the present disclosure provide a means to avoid excessive processing or collection of data or notifications about the user's vehicle, based on a context of the user's proximity to the vehicle, by use of multiple data and signal checks.

A fourth use case involves a user approaching a vehicle and the vehicle having a guard mode enabled, which records content leading up to the user entering the vehicle. Reliance only on a wireless signal of a device associated with the user's location may result in excessive data collection and data processing up until the user enters the vehicle. The systems and methods of this disclosure combine the wireless signal detection with an analysis of information collected by one or more other sensors to determine if further recording and analysis is needed as the user approaches the vehicle (e.g., the vehicle will unlock without a substantial amount of sensor data review as the user approaches).

For each of the aforementioned use cases, the systems and methods of the disclosure provide two or more inputs which are analyzed to determine how much processing or data collection needs to be provided by a vehicle security system to enable seamless ingress and egress of a user recognized by the vehicle (e.g., determining whether a first sensor, such as a wireless signal receiver, has collected enough data to lock or unlock a vehicle and, in response to determining the first sensor data is insufficient, using data from a second sensor such as a camera to determine whether instructions to lock or unlock the vehicle should be generated). These approaches provide clarity for the vehicle systems regarding a context of a user proximity relative to the vehicle and enable more efficient data processing for a preferred user experience with the vehicle. For example, when determining whether to lock or unlock vehicle doors, both video data and RF signal data may be utilized to confirm whether a user is approaching or departing the vehicle.

DETAILED DESCRIPTION

Methods and systems are provided herein for a vehicle that uses at least one of sensor data or wireless signals to determine a vehicle state and corresponding vehicle responses to at least one person or object approaching a vehicle.

The methods and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable medium. Computer-readable medium includes any medium capable of storing data. The computer-readable media may be transitory, including, but not limited to, propagating electrical or electromagnetic signals, or may be non-transitory including, but not limited to, volatile and non-volatile computer memory or storage devices such as a hard disk, floppy disk, USB drive, CD, media cards, register memory, processor caches, Random Access Memory (RAM), etc.

FIG.1depicts scenario100for determining a vehicle state, in accordance with some embodiments of the disclosure. Scenario100comprises elements which may be incorporated, in whole or in part, into each of scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Scenario100may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Scenario100may involve the use of any element or all elements in vehicle800ofFIG.8. Scenario100may also incorporate activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Scenario100is comprised of object102(e.g., a user), which is coupled to user device104(e.g., the user is holding user device104). User device104generates wireless signal106. Wireless signal106may be comprised of an ultra-wideband signal, a Bluetooth signal, a radio frequency (e.g., “RF”) signal, or any suitable communication signal configured to transmit data, without a physical communication channel (e.g., a wire), corresponding to one or more of user102or user device104. Additionally, wireless signal106may correspond to user identification data of a user associated with user device104. For example, the user identification data may be used by one or more of processing circuitry118and system state controller116to determine that object102is the user of vehicle112. One or more of processing circuitry118and system state controller116may be integrated into or embedded in a central gateway module of the vehicle (e.g., a module configured to transfer or transmit data across one or more vehicle networks to one or more separate modules), a body control module, an electronic control unit communicatively coupled to one or more vehicle networks or modules, or any combination thereof. Wireless signal106is characterized by one of signal strengths108A and108B, depending on whether object102is following departing path110A or approaching path110B. For example, signal strength108A shows a decline in a magnitude associated with wireless signal106as object102and user device104depart from vehicle112along departing path110A. In contrast, signal strength108B shows an increase in a magnitude associated with wireless signal106as object102and user device104approach vehicle112along approaching path110B.

Signal strengths108A and108B may be measured, determined, or characterized by sensor114arranged within vehicle112. Sensor114interfaces with at least one of system state controller116and processing circuitry118by transmitting data related to wireless signal106. In some embodiments, multiple of sensor114are arranged at different locations on, in, or around the vehicle. By comparing signal strengths received via different iterations of sensors114, the general location of the user can be determined relative to the vehicle. This general location can then be analyzed by object detection sensors to see if there is an object at that location and, more particularly, can be used to determine that the object is the user and the user is one or more of within a particular vehicle proximity, approaching the vehicle, or departing the vehicle.

System state controller116determines which system states of vehicle112, or systems comprising vehicle112, are active. An exemplary vehicle state management system is depicted in detail inFIG.10. As shown in scenario100, a pair of example vehicle system states are related to whether sensors are enabled or disabled and whether the vehicle doors are locked or unlocked. For example, if object102is determined to be leaving a proximity of vehicle112, then system state controller116may interface with processing circuitry118to transmit instructions throughout various modules controlling devices and systems of vehicle112to enable sensors that characterize an environment around vehicle112(e.g., to detect and track objects) and also may transmit instructions to lock all vehicle doors since there is no user around vehicle112. In another example, if object102is determined to be approaching a proximity of vehicle112, then system state controller116may interface with processing circuitry118to transmit instructions throughout various modules controlling devices and systems of vehicle112to disable sensors that characterize an environment around vehicle112and also may transmit instructions to unlock all vehicle doors since there is a user approaching vehicle112.

Object102is detected by one or more of sensors120A and120B. Sensors120A may be comprised of a camera, a lidar sensor, a radar sensor, a thermal sensor, an ultrasonic sensor, or any sensor suitable for capturing data (e.g., related to distance or presence of objects relative to one or more of a user or a vehicle), or a combination thereof. Sensors120A may be used to define vehicle proximity122(e.g., up to 20 feet away from vehicle112). For example, sensors120A may have a predefined range of detection of objects (e.g., object102). Depending on what type of object crosses vehicle proximity122, system state controller116may modify the vehicle state such that doors are unlocked, or data collected by sensors120A and120B may not be processed or stored. The type of object detected may be determined by object detection process900ofFIG.9. If object102is determined to be the user of vehicle112, then one or more of system state controller116and processing circuitry118may determine to lock or unlock the vehicle depending on whether the user is determined to be approaching or leaving the vehicle. Sensors120B may comprise a camera, a lidar sensor, a radar sensor, a thermal sensor, an ultrasonic sensor, or any sensor suitable for capturing data (e.g., capable of providing one or more matrices of information to display detected objects and details thereof), or a combination thereof. Sensors120B provide additional data (e.g., visual data via video frames) for determining the type of object that object102is. Additionally, sensors120B may be utilized to determine whether object102is approaching or leaving vehicle112based on movements of object102between vehicle proximity122and vehicle proximity124(e.g., up to 5 feet away from vehicle112). In some embodiments, the relative position of object102to vehicle proximity122and vehicle proximity124may result in activation or deactivation of one or more of sensors120A, and sensors120B. For example, if object102is determined to be the user of vehicle112and is detected within vehicle proximity122and is within vehicle proximity124, both sensors120A and120B may be deactivated to avoid additional processing. In another example, if object102is determined to not be the user (e.g., a person not affiliated with vehicle112) and is detected within vehicle proximity122and vehicle proximity124, then both sensors120A and120B may remain active with the doors of vehicle112remaining locked so as to capture the movements of object102and alert a user of the vehicle to the presence of object102. A third example involves object102being determined to be the user of vehicle112and the user remains within vehicle proximity122without entering vehicle proximity124. In this example, sensors120A may remain active while sensors120B are deactivated since the user can see what is occurring around vehicle112.

Sensor114, sensors120A, and sensors120B all collect data which are used to determine a vehicle state. Sensor114is used to collect data to determine a signal strength of wireless signal106. The combination of sensors120A and120B provide additional data to address inconsistencies with what object102is doing relative to vehicle112by providing additional characterizations of object102which would otherwise be absent or lacking by relying only on data from sensor114. For example, sensor114relies only on data related to wireless signal106which may have attenuated data points caused by interference between wireless signal106and sensor114(e.g., user device104is in a pocket of a user corresponding to object102). If system state controller116or processing circuitry118only rely on data from sensor114, then an improper determination of whether object102is approaching or leaving one or more of vehicle112, vehicle proximity122, and vehicle proximity124may be made resulting in the doors of vehicle112being unlocked when a user is leaving vehicle112and vehicle proximity122. Since sensors120A and120B provide additional data (e.g., inclusive of video footage capturing movements of object102), system state controller116and processing circuitry118can utilize the additional data to verify the movements of object102and determine whether the vehicle state is compatible with the signal strength of wireless signal106and the current position or movements of object102.

In some embodiments, system state controller116and processing circuitry118interface to modify a vehicle state based on a determined user state. For example, where object102is determined to be the user, a location of object102is determined. The location data associated with object102(e.g., data indicating whether the user is inside or outside the vehicle, whether the user is within one or more of vehicle proximities122and124), as collected by one or more of sensors120A,120B, and114, is analyzed for determining whether a current vehicle state is consistent with the location of object102. For example, if signal strength108A indicates a user is leaving vehicle112while sensor data collected by sensors120A and120B indicates the user remains within vehicle proximity124, then processing circuitry118interfaces with system state controller116to prevent the doors of vehicle112from locking until the user actually leaves vehicle proximity124. The vehicle state is determined based on the signal strength and sensor data related to the detected object such that a vehicle user and vehicle112are not subjected to doors being locked or power being drained from one or more of notification generation, data collection, and data processing. In some embodiments, the vehicle state is modified in response to a determination that that current vehicle state is inconsistent with a user state corresponding to the user location relative to the vehicle (e.g., doors that are unlocked may be locked when sensor114collects data indicating object102is approaching while sensors120A and120B collect data indicating object102is leaving).

FIG.2depicts scenario200where signal strength202of wireless signal106from user device104and data related to object102detected by one or more of sensors120A and120B, are used to determine a vehicle state, in accordance with some embodiments of the disclosure. Scenario200comprises elements which may be incorporated, in whole or in part, into each of scenario100ofFIG.1, scenario300ofFIG.3, and scenario400ofFIG.4. Scenario200may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Scenario200may involve the use of any element or all elements in vehicle800ofFIG.8. Scenario200may also incorporate activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Scenario200is comprised of object102(e.g., a user), which is coupled to user device104(e.g., the user is taking user device104out of a pocket). Object102is following departing path204such that wireless signal106is attenuated based on interference caused by object102before being received by sensor114of vehicle112. For example, user device104may be in a pocket, a bag, a container, or enclosure associated with object102, resulting in an attenuation of wireless signal106until user device104is removed from the pocket, bag, container, or enclosure. Once user device104is removed, wireless signal106as received by sensor114results in data that may be processed and indicate an incorrect user state (e.g., processing circuitry118determines object102is approaching vehicle112instead of departing vehicle proximity122). In another example, as a result of the positioning of a body of object102between user device104and vehicle112, and by extension sensor114, signal strength202depicts an attenuation of wireless signal106such that a reliance only on data corresponding to wireless signal106by processing circuitry118would yield a false determination that object102is approaching vehicle112. As a result, processing circuitry118may transmit instructions to modify a vehicle state to unlock the doors of vehicle despite the presence of non-user objects206A-206D that are within vehicle proximity122. By incorporating analysis and processing of data collected by one or more of sensors120A and120B, in addition to the processing of data collected by sensor114, processing circuitry118can determine that object102has departed vehicle proximity122(e.g., sensor120B may comprise cameras which can record frames depicting that object102is beyond the threshold of vehicle proximity122). The processing of data collected by sensor114with the processing of data collected by sensors120A and120B prevents a modification of a vehicle state based only on the analysis of one data set by one sensor which may result in an undesired modification of a vehicle state such as unlocking doors when a user is outside vehicle proximity122when signal strength202falsely correlates to object102approaching vehicle122.

FIG.3depicts scenario300where a user of vehicle112remains within close proximity of the vehicle, in accordance with some embodiments of the disclosure. In scenario300, as the user changes positions (e.g., by moving or walking) around vehicle112(e.g., positions302A-302D), the wireless signal106from user device104received by one or more sensors114may vary in signal strength or stability, making it difficult for vehicle112to determine whether the user is approaching or leaving the vehicle. Accordingly, the use of object detection by sensors120A and120B can be used to help determine whether the user remains within vehicle proximity122. By using data from one or both of sensor114and sensors120A and120B, the location of the user can be more accurately determined, and the location can be used to determine whether a vehicle state of vehicle112should be modified (e.g., locked doors should be unlocked or vice-versa).

Scenario300comprises elements which may be incorporated, in whole or in part, into each of scenario100ofFIG.1, scenario200ofFIG.2, and scenario400ofFIG.4. Scenario300may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Scenario300may involve the use of any element or all elements in vehicle800ofFIG.8. Scenario300may also incorporate activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Scenario300is comprised of user304remaining within vehicle proximity122by moving between positions302A-302D. User304corresponds to object102ofFIG.1and object102is determined to be a user of vehicle112based on data processing of data collected by one or more of sensors120A,120B, and114. Thus, in scenario300, user304is object102. Considering that user102remains within proximity122, the vehicle state should reflect that the user has easy access to vehicle112, such as within a garage or at a tailgating event. As a result, notifications generated by processing circuitry118and additional data processing of objects that are different from user304are not required to determine whether vehicle112requires a vehicle state change. As a result, by using one or more of sensors120A,120B, and114, processing circuitry118may determine that the vehicle state of vehicle112may not require locked doors and the vehicle state should not require a full activation of a guard function (e.g., one or more of a limitation of data collected and data processed is reasonable without risking the integrity of vehicle112).

By using one or more of sensors120A,120B, and114to analyze one or more of the position of user304, user device104, and wireless signal106, vehicle112may not use as much processing power and user device104does not receive excessive notifications that contain information the user102can ascertain without the use of the processing of vehicle112(e.g., via processing circuitry118). In some embodiments, one or both of the position of user304and the position of user device104can be determined based on data collected from one or more of sensors120A,120B, and114. Additionally, wireless signal106may comprise characteristics or may comprise data indicative of a position of one or more of object102and user device104.

FIG.4depicts scenario400where wireless signal106(e.g., from user device104) and object102(e.g., as detected by one or more of sensors120A,120B, and114, and determined to be the user of vehicle112with user device104) yield data that, when processed, indicates the user of vehicle112is approaching vehicle112. In response to determining the user of vehicle112is approaching vehicle112, the vehicle state is compared to the user state. For example, the user state may be “user approaching” and the vehicle state may have locked doors. In response to determine the user state is “user approaching,” then the vehicle state is modified such that the vehicle doors are unlocked (e.g., a guard mode that locks the vehicle doors is adjusted such that one or more of the doors being unlocked and a reduction of data processing occurs). In some embodiments, scenario400results in a modification of a vehicle state of vehicle112such that vehicle112is in a vehicle monitoring state focused on data related to user304and wireless signal106.

Scenario400comprises elements which may be incorporated, in whole or in part, into each of scenario100ofFIG.1, scenario200ofFIG.2, and scenario300ofFIG.3. Scenario400may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Scenario400may involve the use of any element or all elements in vehicle800ofFIG.8. Scenario400may also incorporate activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Scenario400is comprised of object102(e.g., a user of vehicle112) following approaching path110B towards vehicle112. Object102includes user device104, which transmits wireless signal106. Wireless signal106has a signal strength which is recorded over time, as depicted by the chart of signal strength108B. Signal strength108B corresponds to data collected by sensor114. Sensors120A and120B affirm that signal strength108B indicates object102is approaching vehicle112. As a result, processing circuitry118modifies the vehicle state to unlock the vehicle doors of vehicle112and may perform one or more of deactivating a guard mode and reducing data processing related to object102.

FIG.5is a flow chart of method500for determining a vehicle state based on a signal strength and a detected object, in accordance with some embodiments of the disclosure. Method500may be executed as a result of any or all of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Method500may be executed with any or all of method600ofFIG.6(e.g., step512ofFIG.5may be executed in response to executing step602ofFIG.6) and method700ofFIG.7(e.g., step508ofFIG.5may be incorporated into step702ofFIG.7), in whole or in part. Method500may be executed using any element or all elements in vehicle800ofFIG.8. Method500may also be incorporated into the activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

At502, a location of the user relative to the vehicle (e.g., vehicle112ofFIG.1) is determined. If the user is determined to be inside the vehicle and not external to the vehicle (NO at502), the location of the user is continued to be monitored. If the user is determined to be external to the vehicle (YES at502), a signal strength of a wireless signal (e.g., wireless signal106ofFIG.1) received from a user device (e.g., user device104ofFIG.1) is determined at504. At506, an object is detected based on a sensor (e.g., one or more of sensors120A and120B ofFIG.1) configured to collected data corresponding to an environment external to a vehicle. The object may be characterized, for example, via data processing scenario900ofFIG.9which relies on data collected by one or more of sensors120A,120B, and114. If the object is not determined to the be the user (NO at508), then an object is detected based on a sensor at506. If the object is determined to be the user (YES at508), then a vehicle state is determined at510, based on the signal strength and the detected object. Example vehicle states are shown inFIG.10, where a vehicle power state and a vehicle security state are modified based on varying inputs received by the vehicle and various messages transmitted between one or more of modules or processing units within the vehicle. In some embodiments, the vehicle state results in a vehicle door lock status that has locked vehicle doors or unlocked vehicle doors.

At512, the current vehicle state is compared to a user state. The user state corresponds to a user location and/or movement relative to the vehicle. For example, the user state may be determined to be “user approaching vehicle” or “user departing vehicle.” If the current vehicle state is consistent with a user state corresponding to a user location relative to the vehicle (NO at512), then the process ends as the vehicle state does not need to be modified. For example, when the current vehicle state corresponds to locked vehicle doors and the user state is determined to be outside a detectable vehicle proximity (e.g., vehicle proximity122ofFIG.1), then the current vehicle state is not modified. If the current vehicle state is inconsistent with a user state corresponding to a user location relative to the vehicle (YES at512), then the current vehicle state is modified at514. For example, if the user is determined to be within a detectable vehicle proximity (e.g., within vehicle proximity124ofFIG.1), the user is approaching the vehicle, and the current vehicle state corresponds to locked vehicle doors, then the vehicle state is modified to unlock the doors (e.g., as shown inFIG.6).

FIG.6is a flow chart of method600for modifying a vehicle state, in accordance with some embodiments of the disclosure. Method600may be executed as a result of any or all of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Method600may be executed with any or all of method500ofFIG.5and method700ofFIG.7(e.g., step602ofFIG.6may be incorporated into step702ofFIG.7), in whole or in part. Method600may be executed using any element or all elements in vehicle800ofFIG.8. Method600may also be incorporated into the activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Method600starts with processing data from one or more sensors (e.g., one or more of sensors114,120A, and120B) to identify an object within a proximity (e.g., within 5 feet or 20 feet) of a vehicle as a user of the vehicle. In some embodiments, the object is determined to be the user. At602, a user state is determined based on one or more of a signal strength (e.g., a signal strength of wireless signal106ofFIG.1) and a location of the user relative to the vehicle (e.g., based on sensor data processed by processing circuitry118ofFIG.1, as collected by one or more of sensors120A,120B, and114). At604, a vehicle state is determined based at least in part on the signal strength and one or more of a detected object or the user. Example vehicle states are depicted inFIG.10, depending on a vehicle power state and messages transmitted relative to a vehicle security state (e.g., a guard mode). At606, the vehicle state is compared to the user state (e.g., using processing circuitry118ofFIG.1). If the current vehicle state is consistent with a user state corresponding to a user location relative to the vehicle (NO at608), then the method ends as the vehicle state complies with the user state and user location (e.g., the user is outside a detectable vehicle proximity and the vehicle doors are locked). If the current vehicle state is inconsistent with a user state corresponding to a user location relative to the vehicle (YES at608), then the current vehicle state is modified at610. Modifying the current vehicle state at610results in the occurrence of any or all of610A,610B, and610C.

610A corresponds to modifying the current vehicle state by preventing generation of notifications that would otherwise be generated based on data collected from the sensor. For example, the vehicle state may result in sensors120A and120B ofFIG.1remaining active such that data collected is processed by processing circuitry118ofFIG.1. Processing circuitry118may be configured to transmit notifications related to vehicle proximity alerts regarding objects approaching vehicle112ofFIG.1to user device104. If it is determined that the user is within vehicle proximity124, the generation and transmission of notifications related to objects approaching the vehicle may be considered superfluous, considering the user's ability to see the environment surrounding the vehicle, and thus are prevented.

610B corresponds to deactivating a guard function that operates on data collected from the sensor (e.g., one or more of sensors120A and120B). For example, a current vehicle state may activate any or all of sensors120A and120B ofFIG.1for collecting data corresponding to the environment surrounding the vehicle. If the user is determined to be within vehicle proximity124ofFIG.1, then the use of any or all of sensors120A and120B is not required for a user to comprehend the environment surrounding vehicle112ofFIG.1. Therefore, modifying the current vehicle state at610B deactivates sensors120A and120B for the duration that the user remains within vehicle proximity124ofFIG.1to prevent power consumption by vehicle112, to maintain operation of guard related functions, and to prevent processing of data by vehicle112, which may result in notifications on user device104ofFIG.1that a user does not require.

610C corresponds to modifying the current vehicle state such that the user is continued to be monitored based on data collected by the sensor. For example, the object is determined to the user (e.g., via data processing scenario900ofFIG.9) and the user proximity to vehicle112ofFIG.1allows the various sensors of vehicle112(e.g., one or more of sensors120A,120B, and114) to continue to collect data related to the user. As with610B, the modification of the vehicle state at610C is configured to prevent power consumption by vehicle112to maintain operation of guard related functions and to prevent processing of data by vehicle112, which may result in notifications on user device104ofFIG.1that a user does not require. As a result, the modification of the vehicle state at610C limits data processing of data collected by sensors of vehicle112to data related to the user such that the vehicle state remains consistent with the user's proximity to the vehicle. In some embodiments, one or more of610A-610C may be performed as part of a vehicle state modification. Different scenarios (e.g., as shown inFIGS.1-4) may result in one or more of the modifications described in reference to610A-610C being performed.

FIG.7is a flow chart method700for modifying a vehicle state based on a comparison of signal parameters, in accordance with some embodiments of the disclosure. Method700may be executed as a result of any or all of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Method700may be executed with any or all of method500ofFIG.5and method600ofFIG.6, in whole or in part. Method700may be executed using any element or all elements in vehicle800ofFIG.8. Method700may also be incorporated into the activation of data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

At702, a first signal parameter of the wireless signal (e.g., wireless signal106ofFIG.1) is determined at a first time. For example, the first signal parameter may be a magnitude of the signal, a wavelength of the signal, or any signal parameter detectable via sensor114ofFIG.1that can be used to determine whether user device104ofFIG.1is approaching or leaving one or more of vehicle112, vehicle proximity124, and vehicle proximity122. At704, a second signal parameter of the wireless signal is determined at a second time. The second signal parameter may be the same parameter type as the first signal parameter, or may be related to the first signal parameter in a manner that enables processing circuitry118ofFIG.1to determine whether user device104ofFIG.1is approaching or leaving one or more of vehicle112, vehicle proximity124, and vehicle proximity122. At706, the first signal parameter is compared to the second signal parameter. If the first signal parameter is determined to not be different (e.g., equal to or similar, such as within a predefined bandwidth of values such as less than a 25% difference in values associated with each parameter) than the second signal parameter (NO at708), then the method ends as the vehicle state is not required to be modified. If the first signal parameter is determined to be different (e.g., outside a predefined bandwidth of values such as more than a 25% difference in values associated with each parameter) than the second signal parameter (YES at708), then a first user state of a user associated with a user device that is a source of the wireless signal is determined at710, based on the difference between the first signal parameter and the second signal parameter.

For example, if the difference is a negative value, then processing circuitry (e.g., processing circuitry118ofFIG.1) may determine the first signal parameter is smaller than the second signal parameter and indicates the object (e.g., the user) is approaching the proximity of the vehicle. In another example, if the difference is positive, then processing circuitry may determine the object is departing the vehicle. As shown inFIG.1, signal strengths108A and108B are depicted as a pair of charts with signal parameter data over time. Signal strength108A depicts that over time the magnitude of the signal strength declines while signal strength108B depicts that over time the magnitude of the signal increases. Each chart corresponds to data processing via method700that would result in different vehicle state modifications, as depicted in system state controller116ofFIG.1.

In some embodiments, the difference in signal strengths may be compared to one or more thresholds. For example, if the difference in signal strengths is less than a lower threshold then a preliminary determination that a user is departing the vehicle may be generated, causing additional sensor data to be reviewed to confirm that the user is departing (e.g., via proximity sensor data analysis or video frame analysis). In another example, if the difference in signal strengths is greater than a higher threshold that is larger in magnitude than the lower threshold, then a preliminary determination that the user is approaching the vehicle may be generated, causing additional sensor data to be reviewed to confirm that the user is approaching.

At712, data collection from a second sensor is activated. For example, one or more of sensors120A and120B ofFIG.1may be activated to collect additional data (e.g., a camera turns on). At714, a second user state of the user associated with the user device that is a source of the wireless signal based on data from the second sensor is determined. For example, frames of video may be analyzed to identify the user and determine if the user is approaching or departing the vehicle. At716, the first user state is compared to the second user state. If the first user state is different from the second user state (YES at718) then the vehicle state is modified at720based on the second user state. For example, the first user state may be determined based on signal noise or RF signal attenuation causing a false or improper determination of whether a user is approaching or departing. Analyzing data from one or more alternative sensors (e.g., proximity sensors or cameras) provides the system with a redundant check to verify the accuracy of an initial user state determination. If the first user state is not different from the second user state (NO at18), then the vehicle state is modified at722based on a difference between the first signal parameter and the second signal parameter. As a continuation of the signal strength example provided earlier, where the difference is negative, the user may be determined to be approaching the vehicle and locked vehicle doors are unlocked. Alternatively, where the difference is negative, the user may be determined to be departing the vehicle and unlocked doors are locked.

FIG.8depicts system800, in accordance with some embodiments of the disclosure. System800comprises elements which may be incorporated, in whole or in part, into vehicle112of each of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. System800may be configured to execute any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. System800may also being incorporated into data processing scenario900ofFIG.9and may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

System800is comprised of vehicle body802and user device804. User device804corresponds to user device104ofFIG.1and is configured to transmit a wireless signal (e.g., wireless signal106ofFIG.1) for data collection by first sensor806. First sensor806corresponds to sensor114ofFIG.1. First sensor806is configured to transmit data to system state controller808(e.g., system state controller116ofFIG.1) and processing circuitry814(e.g., processing circuitry118ofFIG.1). System state controller808and processing circuitry814are also communicably coupled to each other in order to regulate vehicle state changes associated with vehicle body802and systems therein (e.g., locking or unlocking vehicle doors, and activating or deactivating guard functions which monitor environments surrounding vehicle body802). Second sensor810corresponds to one or more of sensors120A and120B ofFIG.1. Second sensor810collects data related to object812and is communicably coupled to processing circuitry814. Processing circuitry814uses data from first sensor806and second sensor810to determine whether system state controller808should transmit instructions to modify a vehicle state of vehicle body802(e.g., modify a door lock status or guard function activation status).

FIG.9depicts data processing scenario900, in accordance with some embodiments of the disclosure. Data processing scenario900comprises elements which may be incorporated, in whole or in part, into each of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Data processing scenario900may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Data processing scenario900may involve the use of any element or all elements in vehicle800ofFIG.8. Data processing scenario900may also utilize vehicle state management system1000ofFIG.10, in whole or in part.

Data processing scenario900is comprised of object102being detected by second sensor810(e.g., one or more of sensors120A and120B ofFIG.1) and user device104transmitting a wireless signal (e.g., wireless signal106ofFIG.1) to first sensor806(e.g., sensor114ofFIG.1). First sensor806and second sensor810are arranged within vehicle112. Vehicle112is within environment902, which extends at least as far as vehicle proximity122ofFIG.1. First sensor806collects first sensor data904and second sensor collects second sensor data906. Each of first sensor data904and second sensor data906are transmitted to processing circuitry908(e.g., processing circuitry118ofFIG.1) and stored in data queue910. Data queue910transmits first sensor data904and second sensor data806to machine learning model912. Machine learning model912includes a library of known objects as characterized by previously collected data and includes a capability to identify new objects where one or more of first sensor data904and second sensor data906do not corroborate to previously identified objects. Based on the processing accomplished via machine learning model912, processing circuitry908outputs object determination914(e.g., identifies object102as the user of vehicle112), which is then used to perform vehicle state processing916. Vehicle state processing916corresponds to the modification of vehicle states as described in relation to scenarios100-400ofFIGS.1-4. In some embodiments, vehicle state processing916is executed via vehicle state management systems1000ofFIG.10.

FIG.10depicts vehicle state management system1000, in accordance with some embodiments of the disclosure. Vehicle state management system1000comprises elements which may be incorporated, in whole or in part, into each of scenario100ofFIG.1, scenario200ofFIG.2, scenario300ofFIG.3, and scenario400ofFIG.4. Vehicle state management system1000may result in the execution of any or all of method500ofFIG.5, method600ofFIG.6, and method700ofFIG.7, in whole or in part. Vehicle state management system1000may involve the use of any element or all elements in vehicle800ofFIG.8. Vehicle state management system1000may also incorporate activation of data processing scenario900ofFIG.9, in whole or in part.

Vehicle state management system1000comprises vehicle power state progression1002and vehicle security state progression1004. Vehicle power state progression1002provides an example of how vehicle112ofFIG.1may change between power states in order to engage or disengage a vehicle security state (e.g., a vehicle state enabling activation of a series of guard functions as described in reference toFIG.1). Vehicle power state progression1002is comprised of vehicle sleep power state1006, vehicle standby power state1008, vehicle monitor power state1010, and vehicle active power state1012. Vehicle sleep power state1006deactivates or disengages a vehicle security state as there is no power available from the vehicle to enable the vehicle security state. When initial input1014is received (e.g., sensor114ofFIG.1detects user device104or a key FOB is detected), vehicle sleep power state1006is modified to vehicle standby power state1008, where the vehicle is now capable of activating systems within the vehicle in response to certain inputs (e.g., object102ofFIG.1is detected by one or more of sensors120A,120B, and114). When monitor confirmation1016is received (e.g., via system state controller116ofFIG.1in response to determining one or more of sensors120A,120B, and114are active and collecting data), vehicle standby power state1008is modified to vehicle monitor power state1010, which enables vehicle security state related systems of the vehicle to receive power, collect data, and transmit information between vehicle modules. When start command1018is received, vehicle standby power state1008is modified to vehicle active power state1012such that systems including a vehicle powertrain are active. The vehicle security state may be modified, activated, or deactivated, depending on the use of the vehicle when in vehicle active power state1012(e.g., some or all of guard functions may remain active when the powertrain is active such that a user is alerted to objects in the environment surrounding the vehicle).

Vehicle power state progression1002includes a series of power state modification protocols to reduce the power required by the vehicle to operate in each state. For example, vehicle active power state1012is modified back to vehicle monitor power state1010in response to receiving cease powertrain operation instruction1020(e.g., a button is pressed, or knob is turned to deactivate the vehicle powertrain). Vehicle monitor power state1010may be modified back to vehicle standby power state1008in response to timeout1022being achieved (e.g., a predetermined amount of time has passed without receiving data or an instruction) due to a lack of reception of monitor confirmation1016(e.g., the vehicle security state is not activated and one or more of sensors120A,120B and114do not collect data). In response to timeout1022being achieved and the vehicle power state being modified back to vehicle standby power state1008, the vehicle monitors signals for initial input1014. Vehicle standby power state1008may be modified back to vehicle sleep power state1006in response to timeout1024being achieved (e.g., a predetermined amount of time has passed without receiving data or an instruction) due to a lack of reception of initial input1014(e.g., none of a user, a user device, or vehicle key fob are detected).

Vehicle security state progression1004is configured to be modified based at least in part on the modification of vehicle power states according to vehicle power state progression1002such that the systems active during the progression of vehicle states aligned with vehicle security state progression1004receive enough power to provide a vehicle user with appropriate information and function. Additionally, vehicle security state progression1004may be modified based on a determination that a user is departing or approaching the vehicle. For example, signal strength analysis may trigger an initial power up of secondary sensors, which then receive power and provide data for processing to confirm a user state (e.g., approaching or departing).

Disengaged security state1026corresponds to vehicle sleep power state. In response to initial input1014being received to modify the vehicle power state to vehicle standby power state1008, activation command1028is received resulting in a modification of disengaged security state1026to engaged security state1030. Engaged security state1030yields monitor confirmation1016which modifies the vehicle power state to vehicle monitor power state1010. This enables the vehicle to modify the states of one or more of proximity sensors, cameras, and remote device wireless signal detectors, as represented by proximity sensor security state1032, camera security state1034, and wireless signal detection security state1036. Any or all of these states result in data being collected and transmitted to processing circuitry (e.g., processing circuitry118ofFIG.1) for the enablement of data processing state1038. Data processing state1038interfaces with auxiliary system controller1040, which is configured to at least lock and unlock vehicle doors, as shown by door locks engaged security state1042and door locks disengaged state1044. Each of the security states shown in vehicle security state progression1004may be deactivated in response to instructions transmitted via data processing state1038or in response to a reduction in power depicted via vehicle power state progression1002.

The systems and processes discussed above are intended to be illustrative and not limiting. One skilled in the art would appreciate that the actions of the processes discussed herein may be omitted, modified, combined, and/or rearranged, and any additional actions may be performed without departing from the scope of the invention. More generally, the above disclosure is meant to be exemplary and not limiting. Only the claims that follow are meant to set bounds as to what the present disclosure includes. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods.

While some portions of this disclosure may refer to examples, any such reference is merely to provide context to the instant disclosure and does not form any admission as to what constitutes the state of the art.