Selfie button for vehicle cameras with flash

Systems and methods for logging images using integrated cameras, lights, and sensors of a vehicle are provided. The system may include a handheld actuator, e.g., a key fob, having a button for initiating the taking of a photo or the start or stop of the recording of a video by the integrated cameras of the vehicle. Moreover, the actuator may select lighting settings such that the integrated lights of the vehicle may illuminate the vicinity of the vehicle to create a flash effect for a photo, or illuminate the vicinity of the vehicle for a video. The image data captured by the cameras may be at least temporarily stored, and transmitted to, e.g., a mobile device within a predetermined range of the vehicle.

BACKGROUND

People love to log journeys such as road trips. These logs might include photos and artifacts from the trip such as ticket stubs, post cards, etc. Anyone driving down the road knows that taking pictures through a car window or through an open window results in blurry or distorted pictures. People also may not be aware of the best time or location to take a picture to capture a location, landmark, or interesting roadside attraction. Likewise, the selfie has become a staple of evidence of being somewhere. Many selfies are restricted by the length of your arm but other novelty devices have been developed to allow more background in the pictures including selfie sticks, smart phone stands, pocket size selfie drones, etc.

Most modern cars have cameras built in, e.g., both on the front and the rear of the vehicle. In addition, many vehicles are integrating cameras for full 360 viewing as well as cameras directed into the cabin of the vehicle. Currently, these cameras are used for vehicle operations such as backing up, parking, security, and/or monitoring drivers for distracted or sleepy behaviors, but they do not provide access to the vehicle occupants for their own use cases. Moreover, some applications have explored using vehicle cameras in aggregate to find a target vehicle through video recognition software. It is with respect to these and other considerations that the disclosure made herein is presented.

DETAILED DESCRIPTION

Overview

Disclosed are systems and methods for using vehicles to capture images or videos during a road trip. The system may involve adding a photo button to the traditional key fob that allows the key fob to transmit a signal to the vehicle radio receiver and send that signal to the vehicle processor and camera modules to request a photo be captured. Likewise, a video button may be used to send a signal to begin a video capture and pressed a second time to stop the video capture. In both cases a response signal may be sent so the horn of the vehicle could honk, and/or lights of the vehicle could flash to indicate the picture or video had been taken, or in the case of a video, the video captured had ended.

The beginning and ending signals may be different. For example, a single flash may indicate the video capture had begun and a double flash may indicate the video capture had stopped and/or the photo had been captured. Sensors of the car also which allow the vehicle to identify if lighting is low and that headlights would need to be used while driving, may be accessed and timed with the video capture for lighting the surrounding of the video, and in the case of a photo, the bright setting on the headlights may pulse on to provide a flash-like quality to fill the image with lighting. To manually indicate whether lighting is required or not, the key fob photo and video button may be, e.g., pushed twice, sending a different signal indicating to the vehicle processor to time the lighting with the photo or video capture. When a person within proximity of a vehicle uses a photo key fob, the vehicle may also identify devices paired with the vehicle. With an application on the phone to leverage this paring, the vehicle may automatically send the picture and/or video directly to the connected and paired device, for example, after compressing the photo size. Alternatively, a full-size file may be transmitted. If the paired connected device should pass out of range prior to completing the transfer, the transfer could be completed after the device is back in range. Thus, the device may store the amount of data that was successfully transferred through confirmation signals passed wirelessly. In the photo/video settings on the vehicle and/or in the application (which may update between the two when settings are changed) the preferred delivery method may be set by the user. If the direct local delivery method wasn't desired, a text from the vehicle may be sent including the photo from the car with a time stamp associated with the picture. Likewise the photo, may be stored in the cloud on a server for later viewing or download.

The photo capture request may be generated on a mobile phone application with a digital representation of a button and settings for flash and backlighting preferences. This signal may be sent directly between the car and the phone, e.g., via Bluetooth, Wi-Fi, or a radio signal. Sending signals through the cloud via a cellular connection is common for most vehicle-to-smart device interactions, but may not allow real time photo functionality as desired with typical selfie photography. The delivery of the picture may be the same as the key fob photo button.

Moreover, an automatic feature may be used to capture pictures or video as well. For example, vehicle sensors (such as radar, LiDar, sonar, optical, etc.) typically on the vehicle for proximity detection when parking or for automatic cruise control or automatic braking may be used. Radar sensors use relatively low energy compared with cameras and video motion change to identify a person or object entering the view of the camera. Radar sensors are typically positioned in an array on a vehicle and may detect how the relative location of a person/object to the vehicle. Setting the radar sensor in automatic photo or video mode may serve as a motion detector that may actuate the camera to capture an image or photo. In this auto mode where the cameras would know where a person/object approaching the vehicle is located, typical proximity technology to recognize a key fob or smart device of the person, e.g., vehicle owner, may be used to confirm whether the approaching person picked up by the radar auto picture mode is the owner or not. For the purposes of selfies, taking a picture of everything that passes by would use significant data. Thus, correlating the location of a person and the identity of the person through the vehicle paired device would ensure that the vehicle is only taking pictures of the owner/operator as they approach or leave the vehicle. The automatic photos settings may be set to take photos, e.g., upon return or departure of the person from the vehicle.

In addition, radar sensors also may be used to detect the relative direction of motion of an object as it passes through the range of the sensors to allow the proper camera(s) to be triggered, as well as when video and/or photos are captured. This may be used to track a person or object as it moved around the vehicle.

An adaptation of the photo button may allow an owner to select which camera to use when capturing the picture or to use multiple cameras to create a series of pictures that may be used to, e.g., generate a panoramic view or capture a stop-motion action of someone or something moving around the car. This allows a user to choose the best picture or put a series of pictures together to more fully show surroundings.

Illustrative Embodiments

The disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which example embodiments of the disclosure are shown. This disclosure may, however, be embodied in many different forms and should not be construed as limited to the example embodiments set forth herein. It will be apparent to persons skilled in the relevant art that various changes in form and detail can be made to various embodiments without departing from the spirit and scope of the present disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the above-described example embodiments but should be defined only in accordance with the following claims and their equivalents. The description below has been presented for the purposes of illustration and is not intended to be exhaustive or to be limited to the precise form disclosed. It should be understood that alternate implementations may be used in any combination to form additional hybrid implementations of the present disclosure. For example, any of the functionality described with respect to a particular device/component may be performed by another device/component. Further, while specific device characteristics have been described, embodiments of the disclosure may relate to numerous other device characteristics. Further, although embodiments have been described in language specific to structural features and/or methodological acts, it is to be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the embodiments.

Certain words and phrases are used herein solely for convenience and such words and terms should be interpreted as referring to various objects and actions that are generally understood in various forms and equivalencies by persons of ordinary skill in the art.

Referring now toFIG. 1, an exemplary system for logging images is provided. System100may include image logging platform200and vehicle101having one or more cameras102, one or more lights104, e.g., headlights and/or taillights of vehicle101, one or more radar sensors106, and control module108, all integrated with vehicle101. In addition, system100may include handheld actuator110, e.g., a key fob, and/or an application installed on mobile device120. Image logging platform200may be located on one or more servers, e.g., stored on cloud160or on control module108, and communicate with cameras102, lights104, radar sensors106, control module108, actuator110, and/or mobile device120via network150as described in further detail below.

Vehicle101may be a manually driven vehicle (e.g., no autonomy) and/or configured and/or programmed to operate in a fully autonomous (e.g., driverless) mode (e.g., Level-5 autonomy) or in one or more partial autonomy modes which may include driver assist technologies. Examples of partial autonomy (or driver assist) modes are widely understood in the art as autonomy Levels 1 through 4. A vehicle having a Level-0 autonomous automation may not include autonomous driving features. An autonomous vehicle (AV) having Level-1 autonomy may include a single automated driver assistance feature, such as steering or acceleration assistance. Adaptive cruise control is one such example of a Level-1 autonomous system that includes aspects of both acceleration and steering. Level-2 autonomy in vehicles may provide partial automation of steering and acceleration functionality, where the automated system(s) are supervised by a human driver that performs non-automated operations such as braking and other controls. In some aspects, with Level-2 autonomous features and greater, a primary user may control the vehicle while the user is inside of the vehicle, or in some example embodiments, from a location remote from the vehicle but within a control zone extending up to several meters from the vehicle while it is in remote operation. Level-3 autonomy in a vehicle can provide conditional automation and control of driving features. For example, Level-3 vehicle autonomy typically includes “environmental detection” capabilities, where the vehicle can make informed decisions independently from a present driver, such as accelerating past a slow-moving vehicle, while the present driver remains ready to retake control of the vehicle if the system is unable to execute the task. Level-4 autonomous vehicles can operate independently from a human driver, but may still include human controls for override operation. Level-4 automation may also enable a self-driving mode to intervene responsive to a predefined conditional trigger, such as a road hazard or a system failure. Level-5 autonomy is associated with autonomous vehicle systems that require no human input for operation, and generally do not include human operational driving controls. According to embodiments of the present disclosure, image logging platform200may be configured and/or programmed to operate with a vehicle having a Level-4 or Level-5 autonomous vehicle controller.

Cameras102may be standard cameras integrated with vehicle101that capture image data to facilitate vehicle operations such as backing up, parking, security, and/or monitoring drivers for distracted or sleepy behaviors. Accordingly, cameras102may be disposed along the exterior of vehicle101, e.g., on the front, side panels, side view mirror, and/or rear of vehicle101, to capture image data external to vehicle101and/or within the interior of vehicle101, e.g., on the dash, rearview mirror, integrated with control module108, etc., to capture image data of the interior space within vehicle101. Further, additional cameras may be integrated with vehicle101to provide additional fields of view in and around vehicle101. As will be understood by a person having ordinary skill in the art, less or more cameras102may be integrated with vehicle101than is depicted inFIG. 1.

Lights104may be standard lights integrated with vehicle101such as headlights or taillights for illuminating the vicinity of vehicle101, and or additional lights such as side view mirror lights, brake lights, etc. Further, additional lights may be integrated with vehicle101to provide additional illumination around vehicle101. As will be understood by a person having ordinary skill in the art, less or more lights104may be integrated with vehicle101than is depicted inFIG. 1. In addition, vehicle101may include light sensors for detecting the amount of light in the vicinity of vehicle101, such that lights104may be automatically activated upon determination that the vicinity around vehicle101is too dark via the light sensors.

Radar sensors106may be standard sensors integrated with vehicle101for facilitating vehicle operations to detect a proximity of a body, e.g., person or object, in the vicinity of vehicle101to facilitate vehicle operations such as parking, automatic cruise control, or automatic braking of vehicle101. Accordingly, radar sensors106may be disposed along the exterior of vehicle101, e.g., on the front, side panels, side view mirror, and/or rear of vehicle101. In some embodiments, radar sensors106may be other types of sensor, such as sensors that use ultrasound technology to detect the proximity of a body in the vicinity of vehicle101. Moreover, other sensors integrated within the vehicle may be configured to detect handheld actuator110and/or mobile device120within the vicinity of vehicle101and/or directly or over a network, wirelessly communicate with handheld actuator110and/or mobile device120. Further, additional radar sensors may be integrated with vehicle101to provide additional fields of view around vehicle101. As will be understood by a person having ordinary skill in the art, less or more radars106(or other sensors) may be integrated with vehicle101than is depicted inFIG. 1.

Control module108of vehicle101may access the GPS system of vehicle101as well as information stored in an online database regarding, e.g., popular nearby locations based on the geographical location of vehicle101, and suggest and/or direct vehicle101to the popular nearby location. For example, the popular nearby locations may be national monuments or scenic vantage points. Moreover, control module108may use data received from cameras102, radar sensors104, and/or other vehicle sensors to determine whether vehicle101is in a desirable position at the popular nearby location to capture the photo or video, e.g., whether the surrounding view is obstructed or whether the lighting is poor. In some embodiments, control module108may include a graphical user interface (GUI) which incorporates the functionality of actuator110and/or mobile device120. Accordingly, the GUI of control module108may permit an occupant of vehicle101to initiate the taking of a photo or recording of a video, adjust image settings, and/or view photos/videos captures by cameras102. Additionally, control module108may house image logging platform200. Moreover, the GUI of control module108may permit the user to adjust preprogrammed settings of image logging platform200, e.g., automatic feature as described in further detail below, whether to initiate the automatic image capture upon departure or arrival from vehicle101, whether to directly transmit image data to paired mobile devices or via push notification or email, timers, which cameras to use, what style of photo/video to capture, etc.

Handheld actuator110may be, for example, a standard key fob that may be carried by an occupant of vehicle101, e.g., the driver, with additional button(s)112, e.g., a camera and/or video button. Button112may be actuated by a user to send a signal to a processor of vehicle101, e.g., image logging platform200, to initiate the taking of a photo and/or recording of a video by cameras102of vehicle101. As shown inFIG. 1, actuator110may include a single button112. Thus, button112may be actuated in various manners to initiate, e.g., the taking of a photo or the start and stop of the recording of a video. For example, button112may be pressed once to take a photo, twice to start the recording of a video, and once or twice following the start of a video to stop the video recording. As will be understood by a person having ordinary skill in the art, button112may be preprogrammed to initiate the taking of a photo and/or the recording of a video using other actuation patterns. Alternatively, actuator110may include a separate camera button and a separate video button. Accordingly, the camera button may be pressed once to take a photo, and the video button may be pressed once to start the recording of a video and once more to stop the recording of the video. Moreover, button112may be actuated in a predetermined manner to send a signal to vehicle101to cause lights104to create a flash that is timed with the taking of a photo or to illuminate the vicinity of vehicle101that is timed with the recording of a video. Alternatively, actuator110may include a separate button, e.g., a flash button, to send a signal to vehicle101to cause lights104flash/illuminate.

Mobile device120may have an application installed thereon for interfacing with image logging platform200. For example, the application may permit mobile device120to receive image data from image logging platform200, e.g., video data or picture data, captured by cameras102. Mobile device120may receive and store the image data, e.g., when mobile device is within a predetermined range of vehicle101. If mobile device120is in the process of downloading an image data file, and goes out of the predetermined range, downloading of the image data file may be temporarily paused until mobile device120is within the predetermined range of vehicle101. Mobile device120may receive the image data automatically when mobile device120is within a predetermined range of vehicle101, or upon acceptance of a push notification sent to mobile device120by vehicle101. Additionally or alternatively, a user may actively download the image data files from image logging platform200via the application installed on mobile device120or via an email sent to an email account accessible by mobile device120or another computing device. Mobile device120may be, for example, a smartphone, a tablet, or a smartwatch. In one embodiment, handheld actuator110may be integrated with mobile device120. Accordingly, the application may display graphical representations of one or more buttons that may be actuated to send signals to vehicle101to initiate the taking of a photo, recording of a video, or providing illumination as described above.

Network150may include any one, or a combination of networks, such as a local area network (LAN), a wide area network (WAN), a telephone network, a cellular network, a cable network, a wireless network, and/or private/public networks, such as the Internet. For example, network150may support communication technologies, such as TCP/IP, Bluetooth, cellular, near-field communication (NFC), Wi-Fi, Wi-Fi direct, machine-to-machine communication, and/or man-to-machine communication.

Information shared between image logging platform200, cameras102, lights104, radar sensors106, control module108, handheld actuator110, and/or mobile device120, may be stored on cloud storage160and may be bi-directional in nature. For example, in one case, image logging platform information may be transferred from image logging platform200to cloud storage160. Such information stored on cloud storage160may be accessed and downloaded by mobile device120, or other devices, e.g., a remote computing device.

Referring now toFIG. 2, components that may be included in image logging platform200are described in further detail. Image logging platform200may include one or more processors202, communication system204, and memory206. Communication system204may include a wireless transceiver that allows image logging platform200to communicate with cameras102, lights104, radar sensors106, control module108, actuator110, and/or mobile device120. The wireless transceiver may use any of various communication formats, such as, for example, an Internet communications format, or a cellular communications format.

Memory206, which is one example of a non-transitory computer-readable medium, may be used to store operating system (OS)220, key fob module208, mobile application interface processing module210, camera data processing module212, radar data processing module214, light module216, and image logging module218. The modules are provided in the form of computer-executable instructions that may be executed by processor202for performing various operations in accordance with the disclosure.

Key fob module208may be executed by processor202for receiving one or more signals from actuator110, e.g., a key fob. For example, key fob module208may receive one or more signals from actuator110indicative of a command by the user to initiate either the taking of a photo or the start or stop of a recording of video. As described above, actuator110may be actuated in various manners, e.g., various actuation patterns or actuation of separate buttons, depending on whether the action to be taken. Moreover, key fob module208may receive one or more signals from actuator110indicative of desired settings, e.g., whether or not to cause lights104to illuminate the vicinity of vehicle101when a photo or video is captured by cameras102. Accordingly, key fob module208may process the signal received from actuator208to determine which action to take.

Mobile application interface processing module210may be executed by processor202for interfacing with a mobile application installed on mobile device120. For example, mobile application interface processing module210may determine if mobile device120is paired with image logging platform200, e.g., via communication system204, and further may transmit image data to mobile device120when paired. Moreover, mobile application interface processing module210may pause the transmission of image data to mobile device120when it is determined that mobile device120is outside a predetermined range from vehicle101, and may cause the remaining image data to be stored, e.g., either on memory206or on a cloud server, until mobile device120is determined to be within the predetermined range. In some embodiments, mobile application interface processing module210may automatically transmit the image data to mobile device120when paired, or alternatively, mobile application interface processing module210may transmit a push notification to mobile device120to download the image data. Additionally or alternatively, mobile application interface processing module210may transmit the image data via, e.g., email, which may be accessed by mobile device120or another computing device. Moreover, when actuator110is integrated with mobile device120as described above, mobile application interface processing module210may interface with mobile device120to receive the one or more signals from mobile device120indicative of a command by the user to initiate either the taking of a photo or the start or stop of a recording of video, or to adjust settings.

Camera data processing module212may be executed by processor202for causing cameras102to take a photo and/or start and stop the recording of a video responsive to the command signal received and processed by key fob module208. In some embodiments, camera data processing module212may cause specific cameras of cameras102to take the photo or record video based on data received from radar sensors indicative of the position of a person, e.g., based on the position of actuator110relative to vehicle101. For example, if a user carrying actuator110is detected in the vicinity of the front vehicle101by radar sensors106, camera data processing module212may cause cameras102positioned on vehicle101which have a field of view of the front of vehicle101to take the photo or record the video. In addition, camera data processing module212may cause specific cameras of cameras102to take a photo based on the camera settings selected via actuator110, e.g., a panoramic photo or stop motion photo. Moreover, camera data processing module212may receive and process image data received from cameras102. For example, camera data processing module212may compress the image data so that it may be easily transmitted via communication system204to mobile device120.

Radar data processing module214may be executed by processor202for causing radar sensors106to detect the proximity of a person or object in the vicinity of vehicle101, and for receiving data from radar sensors106indicative of the proximity of the person or object in the vicinity of vehicle101. Moreover, radar data processing module214may determine the relative location of actuator110to vehicle101based on data received from radar sensors106, e.g., when a person carrying actuator110walks toward the front of vehicle101. Accordingly, when key fob module208receives a command to take a photo or record video, radar data processing module214may transmit data indicative of the location of actuator110received from radar sensors106to camera data processing module212such that camera data processing module212may instruct only the cameras of cameras102that have a field of view of where actuator110is located relative to vehicle101to take the photo or record video. Moreover, in an automatic settings mode, radar data processing module214may transmit data indicative of the location of actuator110received from radar sensors106to camera data processing module212such that camera data processing module212may instruct the cameras of cameras102that have a field of view of where actuator110is located relative to vehicle101to take the photo or record video automatically.

Light module216may be executed by processor202for causing lights104to illuminate the vicinity of vehicle101. In addition, light module216may cause specific lights of lights104of vehicle101to illuminate in a predetermined manner to indicate a certain action is being taken. For example, light module216may cause lights104to flash once to indicate that a photo was taken or twice to indicate that a video has begun recording, and twice again to indicated that the video recording has stopped. As will be understood by a person having ordinary skill in the art, different predetermined manners of illumination by lights104may be preprogrammed to indicate different actions being taken. The lights of lights104used to illuminate a photo, e.g., create a flash like effect, or video may be different from the lights of lights104used to indicate an action has been taken. For example, the headlights of vehicle101may be used to create the flash effect, whereas brake lights or lights on the side view mirror of vehicle101may be used to indicate that the photo was taken. Additionally or alternatively, the horn of vehicle101may be used to indicate an action has been taken. Accordingly, control module108may cause the horn of vehicle101to honk in a predetermined manner to indicate a certain action is being taken.

Image logging module218may be executed by processor202for storing image data received from camera data processing module212, e.g., in memory206or on a cloud server, and for instructing mobile application interface module210to transmit the image data to mobile device120.

Referring now toFIG. 3, exemplary method300for logging images using vehicle101is described. At step302, an actuation command may be received by, e.g., key fob module208, from handheld actuator110. At step304, camera data processing module212may cause cameras102to capture image data within a vicinity of the vehicle responsive to the actuation command. For example, camera data processing module212may cause cameras102to take a photo or start or stop the recording of video. At step306, image logging module218may store, at least temporary, the image data received by camera data processing module212in, e.g., memory206or on a cloud server. At step308, mobile application interface module210may transmit the stored image data to mobile device120, e.g., when mobile device120is within a predetermined range of vehicle101.

Implementations of the systems, apparatuses, devices, and methods disclosed herein may comprise or utilize one or more devices that include hardware, such as, for example, one or more processors and system memory, as discussed herein. An implementation of the devices, systems, and methods disclosed herein may communicate over a computer network. A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or any combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmission media can include a network and/or data links, which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of non-transitory computer-readable media.

Those skilled in the art will appreciate that the present disclosure may be practiced in network computing environments with many types of computer system configurations, including in-dash vehicle computers, personal computers, desktop computers, laptop computers, message processors, handheld devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, and/or wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both the local and remote memory storage devices.

Further, where appropriate, the functions described herein may be performed in one or more of hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) may be programmed to carry out one or more of the systems and procedures described herein. Certain terms are used throughout the description, and claims refer to particular system components. As one skilled in the art will appreciate, components may be referred to by different names. This document does not intend to distinguish between components that differ in name, but not function.