Patent Description:
In some instances, a video system may erroneously present "stale video. " Stale video generally means old video that have not been updated with new frames. This may be particularly problematic for displays incorporated into vehicle rearview assemblies. Specifically, presenting stale video to a vehicle occupant may pose significant safety risks. However, there are complications in ensuring stale video is not presented by vehicle video systems, because incoming framerates from cameras, such as rearview cameras or backup cameras, are often variable, but the display framerate maintains constant. Thus, the display and the cameras may have different frame rates at any given time. Accordingly, some repeated frames are expected when the frame rate of the display is greater than that of the camera, due to the display advancing to a new frame before a new image is received from the camera.

Document <CIT> discloses a circuit arrangement for processing digital video stream information providing a solution to detect frame freeze based on generated checksums.

The invention is set out in independent system claim <NUM>.

In accordance with the present disclosure, the problems associated with stale video detection for systems where the camera and display may have different frame rates have been substantially reduced or eliminated.

According to one aspect which is claimed, a system is disclosed. The system comprises a camera, a buffer, a display, and a controller. The camera is operable to capture a series of images at a first frame rate. In some embodiments, the first frame rate may be variable. The buffer has a first number of memory storage slots. Additionally, the buffer is operable to: at a frequency of time intervals, sequentially and cyclically store into the memory storage slots, the most current image received from the camera at each interval. The display is operable to present images at a second frame rate, sequentially and cyclically from the memory storage slots. In some embodiments, the second frame rate may be greater than the first frame rate. Accordingly, the first frame rate may be less than the second frame rate. In some embodiments, the first number may be determined based, at least in part, on a highest expected ratio between the first and second frame rates. The controller is operable to: assign hash values to images of each frame displayed by the display; store novel hash values in a memory; maintain a counter; and determine if one or more images in the buffer are stale. In some embodiments, the controller may be further operable to not store repeat hash values in the memory. In some embodiments, the memory may retain only a second number of the stored novel hash values such that only the most recent stored novel hash values are retained. The second number may be greater than or equal to the first number. The counter is maintained such that assigning a repeat hash value increments the counter by one and assigning novel hash values resets the counter to zero. Further, the stale images is determined based, at least in part, on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may be further based, at least in part, on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being displayed by the display. Additionally or alternatively, the display may be operable to display a notification related to the determination of the buffer having one or more stale images.

In accordance with another aspect of the present disclosure, a device is disclosed. In some embodiments, the device may be a vehicular rearview assembly. Further, the device may comprise a display and a controller. The display may be operable to display images at a first frame rate. The controller may be operable to receive images at a second frame rate. In some embodiments, the images may be operably received from a camera having a field of view rearward relative a vehicle in which the device may be disposed. In some embodiments, the first and second frame rates may be different. Additionally, the second frame rate may be variable. Additionally, the controller may be further operable to, at an interval frequency, sequentially and cyclically store the most current received image in a buffer having a first number of memory storage slots. Additionally, the display may be further operable to display images sequentially and cyclically from the memory storage slots. Further, the controller may be operable to assign hash values to images of each frame displayed by the display; store novel hash values in a memory; maintain a counter; and determine if one or more images in the buffer are stale. In some embodiments, the controller may be further operable to not store repeat hash values in the memory. In some embodiments, the memory may retain only a second number of the stored novel hash values such that only the most recent stored novel hash values are retained. The second number may be greater than or equal to the first number. The counter may be maintained such that assigning a repeat hash value increments the counter by one and assigning novel hash values resets the counter to zero. Further, the stale images may be determined based, at least in part, on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may be further based, at least in part, on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being displayed by the display. Additionally or alternatively, the display may be operable to display a notification related to the determination of the buffer having one or more stale images.

In accordance with another aspect, a system is disclosed. The system may comprise a camera, a buffer, a video recorder, and/or a controller. The camera may be operable to capture a series of images at a first frame rate. In some embodiments, the first frame rate may be variable. The buffer may have a first number of memory storage slots. Additionally, the buffer may be operable to: at a frequency of time intervals, sequentially and cyclically store into the memory storage slots, the most current image received from the camera at each interval. The video recorder may be operable to record images at a second frame rate, sequentially and cyclically from the memory storage slots. In some embodiments, the second frame rate may be greater than the first frame rate. Accordingly, the first frame rate may be less than the second frame rate. In some embodiments, the first number may be determined based, at least in part, on a highest expected ratio between the first and second frame rates. The controller may be operable to: assign hash values to images of each frame recorded by the video recorder; store novel hash values in a memory; maintain a counter; and determine if one or more images in the buffer are stale. In some embodiments, the controller may be further operable to not store repeat hash values in the memory. In some embodiments, the memory may retain only a second number of the stored novel hash values such that only the most recent stored novel hash values are retained. The second number may be greater than or equal to the first number. The counter may be maintained such that assigning a repeat hash value increments the counter by one and assigning novel hash values resets the counter to zero. Further, the stale images may be determined based, at least in part, on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may be further based, at least in part, on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being recorded by the video recorder. Additionally or alternatively, the video recorder may be operable to record a notification related to the determination of the buffer having one or more stale images.

In accordance with yet another aspect of the present disclosure, a device is disclosed. In some embodiments, the device may be a vehicular rearview assembly. Further, the device may comprise a video recorder and a controller. The video recorder may be operable to record images at a first frame rate. The controller may be operable to receive images at a second frame rate. In some embodiments, the images may be operably received from a camera having a field of view rearward relative a vehicle in which the device may be disposed. In some embodiments, the first and second frame rates may be different. Additionally, the second frame rate may be variable. Additionally, the controller may be further operable to, at an interval frequency, sequentially and cyclically store the most current received image in a buffer having a first number of memory storage slots. Additionally, the video recorder may be further operable to record images sequentially and cyclically from the memory storage slots. Further, the controller may be operable to assign hash values to images of each frame recorded by the video recorder; store novel hash values in a memory; maintain a counter; and determine if one or more images in the buffer are stale. In some embodiments, the controller may be further operable to not store repeat hash values in the memory. In some embodiments, the memory may retain only a second number of the stored novel hash values such that only the most recent stored novel hash values are retained. The second number may be greater than or equal to the first number. The counter may be maintained such that assigning a repeat hash value increments the counter by one and assigning novel hash values resets the counter to zero. Further, the stale images may be determined based, at least in part, on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may be further based, at least in part, on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being recoded by the video recorder. Additionally or alternatively, the video recorder may be operable to record a notification related to the determination of the buffer having one or more stale images.

These and other aspects, objects, and features of the present disclosure will be understood and appreciated by those skilled in the art upon studying the following specification, claims, and appended drawings. It will also be understood that features of each embodiment disclosed herein may be used in conjunction with, or as a replacement for, features in other embodiments.

In the drawings:
<FIG>: a schematic representation of a video system.

For the purposes of description herein, the specific devices and processes illustrated in the attached drawings and described in this disclosure are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific characteristics relating the embodiments disclosed herein are not limiting, unless the claims expressly state otherwise.

The present disclosure is directed to a video system <NUM>. Accordingly, <FIG> illustrates a schematic representation of an embodiment of video system <NUM>. Video system <NUM> may comprise an imager <NUM>, a buffer <NUM>, a display <NUM>, a controller <NUM>, and/or a video recorder <NUM>. In some embodiments, video system <NUM> may be incorporated into a vehicle.

Imager <NUM> may be any device configured and/or operable to capture light and generate a plurality of corresponding images. For example, imager <NUM> may be a camera. Accordingly, imager <NUM> may be a Semi-Conductor Charge-Coupled Device (CCD) or a pixel sensor of Complementary Metal-Oxide-Semi-Conductor (CMOS) technologies. In some embodiments, the images may be captured in series as a video stream. Thus, the images may be captured in accordance with a first frame rate. The first frame rate may be approximately <NUM> frames per second. In some embodiments, the first frame rate may be substantially constant. In other embodiments, the first frame rate may be variable. In some such embodiments, the first frame rate may vary based, at least in part, on imaging conditions. In some embodiments, the images captured by imager <NUM> may have a field of view corresponding to a scene exterior the vehicle. Accordingly, the field of view may correspond to a field of view traditionally associated with an interior rearview assembly, driver side exterior rearview assembly, passenger side exterior rearview assembly, or back-up camera. Thus, the scene may be rearward and/or to the side relative the vehicle.

Buffer <NUM> comprises a first number of memory storage slots <NUM>. Therefore, buffer <NUM> may be stored in a computer memory, such as random-access memory (RAM). Further, the computer memory may be a non-transitory computer-readable media (CRM). Additionally, buffer <NUM> may be communicatively connected to imager <NUM>. As used herein, "communicatively connected" may mean connected directly or indirectly though one or more electrical components. Thus, buffer <NUM> may be configured and/or operable to receive images from imager <NUM>. Accordingly, buffer <NUM> may be configured and/or operable to sequentially and cyclically store a most current complete image received from imager <NUM> into one of the memory storage slots <NUM> at a time interval. The time interval may have a frequency. In some embodiments, the frequency may be substantially constant. Therefore, at each time interval, the most current image is stored in a storage slot <NUM>. For example, at a first time interval, the most recent image received from imager <NUM> may be a first image. Thus, the first image may be stored in a first storage slot <NUM>-<NUM>. Subsequently, at a second time interval, the most recent image received from imager <NUM> may then be stored in a second storage slot <NUM>-<NUM>. In some instances, this image may again be the first image if buffer <NUM> has not yet completely received a second image from imager <NUM>. Alternatively, this image may be the second image if the second image has been completely received from imager <NUM>. The second image may be a subsequently recorded image relative the first image. This process may repeat itself through subsequent time intervals and storage slots (e.g., <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>,. <NUM>-n) until an image is stored in the last storage slot <NUM>-n of the first number of storage slots <NUM>. Once an image is stored in the in the last storage slot <NUM>-n, at the next time interval, buffer <NUM> may cyclically revert back to the beginning and then store the most current image in first storage slot <NUM>-<NUM>, overwriting the first image.

Display <NUM> may be any device configured to and/or operable to present one or more digital images for viewing by a user or an occupant. In some embodiments, display <NUM> may be configured and/or operable to directly retrieve the images for presentation from buffer <NUM>. Accordingly, display element <NUM> may be of LCD, LED, OLED, micro-LED, plasma, DLP, or another technology. In some embodiments, display element <NUM> may be disposed in a rearview assembly, such as an interior rearview assembly of the vehicle. The images presented by display <NUM> may be presented at a second frame rate. Thus, the images presented by display <NUM> may be a video stream. Additionally, the second frame rate may be substantially constant. In some embodiments, the second frame rate may be approximately <NUM> frames per second. Accordingly, the second frame rate may be substantially greater than the first frame rate. In some embodiments, the second frame rate may be substantially equal to the time interval frequency. In some embodiments, the first number may be determined based, at least in part, on a highest expected ratio between the first and second frame rates. Further, display <NUM> may be communicatively connected to buffer <NUM>. Accordingly, the frames presented by display <NUM> may be retrieved sequentially and cyclically from the storage slots <NUM>. For example, the image stored in the first storage slot <NUM>-<NUM> may be presented in a first frame and, subsequently, the image stored in the second storage slot <NUM>-<NUM> may be presented in a second frame. This may be continued until the image stored in an nth storage slot <NUM>-n is presented in an nth frame. Further, after presenting the image in the nth storage slot <NUM>-n, display <NUM> may cyclically come back to the first storage slot <NUM>-<NUM> and present the image stored therein as an n+<NUM> frame of a video stream presented by display <NUM>, and the process may re-progress through the storage slots <NUM>, and so forth.

Controller <NUM> may be communicatively connected to imager <NUM>, buffer <NUM>, and/or display <NUM>. In some embodiments, controller <NUM> may be configured and/or operable to receive the one or more images from imager <NUM>. In some such embodiments, controller <NUM> may be configured to and/or operable to sequentially and cyclically store the most current complete image received from imager <NUM> into memory storage slots <NUM> at the time interval. In some embodiments, controller <NUM> may be configured and/or operable to provide display <NUM> with the one or more images for presentation. In some such embodiments, controller <NUM> may be configured and/or operable to sequentially and cyclically retrieve the one or more images from the memory storage slots <NUM> and relay them to display <NUM>. Further, controller <NUM> may comprise a memory <NUM> and a processor <NUM>.

Memory <NUM> be a non-transitory computer-readable media (CRM). Accordingly, memory <NUM> may be configured to and/or operable to store one or more instructions, such as one or more algorithms, to provide for the configuration and operation of controller <NUM>. Examples of memory <NUM> include conventional hard disks, solid-state memories, random access memories (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), optical or magnetic disks, dynamic random-access memory (DRAM). In some embodiments, memory <NUM> may comprise buffer <NUM>. In other embodiments, memory <NUM> and buffer <NUM> may be separate entities. Further, memory <NUM> may store one or more algorithm. The algorithm may be configured to and/or operable to detect stale images. To detect the stale images, the algorithm may be configured and/or operable to assign hash values to images added to the memory storage slots <NUM> of buffer <NUM> and/or images of each frame presented by display <NUM>. In some embodiments, the hash may be a cyclic redundancy check hash (CRC).

The assigned hash values may be stored in the memory <NUM>. In some embodiments, controller <NUM> may only store novel hash values in memory <NUM>. Thus, in such an embodiment, controller <NUM> may not store repeat or duplicate hash values in memory <NUM>. Additionally, in some embodiments, controller <NUM> may retain only a second number of the stored hash values. The second number of retained stored hash values, may be only the most recent stored hash values. In some embodiments, the second number may be greater than or equal to the first number. Accordingly, the second number, in some such embodiments, may be equal to the first number.

Additionally, the algorithm may be configured and/or operable to maintain a counter. Assigned hash values that match any previously assigned hash value stored in memory <NUM> increment the counter by one. Conversely, assigned hash values that do not match any previously assigned hash value reset the counter to zero. Further, the algorithm may be configured and/or operable to determine that one or more images in the buffer are stale based, at least in part, on the counter value exceeding the number of frames stored in buffer.

Processor <NUM> may be communicatively connected to memory <NUM>. Further, processor <NUM> may be any device or electronic circuit configured and/or operable to process or execute one or more sets of electronic instructions, such as the algorithm. These instructions may be stored in memory <NUM>. Examples of processor <NUM> may include a central processing unit (CPU), a microprocessor, and/or an application specific integrated circuit (ASIC).

In some embodiments, the algorithm may be operable to prevent stale images from being presented by display <NUM>. Based, at least in part, on the determination of one or more of the images in buffer <NUM> being stale, controller <NUM> may prevent to the images from being presented by display <NUM>. In some such embodiments, controller <NUM> may send an error notification for presentation by display <NUM>.

Video recorder <NUM> may be any device configured and/or operable to record, store, and/or save a series of received images. For example, video recorder <NUM> may be part of a black box for a vehicle. Thus, video recorder <NUM> may comprise a computer memory. The computer memory may be the same as or substantially similar to memory <NUM>. Additionally, video recorder <NUM> may be communicatively connected to imager <NUM>, buffer <NUM>, display <NUM>, and/or controller <NUM>. The received images may be received and saved as a video stream at a third frame rate. The video steam may comprise a series of image frames. Additionally, the third frame rate may be substantially constant. In some embodiments, the third frame rate may be approximately <NUM> frames per second. Accordingly, the third frame rate may be substantially greater than the first frame rate. In some embodiments, the third frame rate may be substantially equal to the time interval frequency. In some embodiments, the first number may be determined based, at least in part, on a highest expected ratio between the first and third frame rates. In some embodiments, the series of received images may be retrieved sequentially and cyclically from the storage slots <NUM> of buffer <NUM>. For example, the image stored in the first storage slot <NUM>-<NUM> may be recorded in video recorder <NUM> in a first frame and, subsequently, the image stored in the second storage slot <NUM>-<NUM> may be recorded in a second frame. This may be continued until the image stored in an nth storage slot <NUM>-n is presented in an nth frame. Further, after recording the image in the nth storage slot <NUM>-n, video recorder <NUM> may cyclically come back to the first storage slot <NUM>-<NUM> and record the image stored therein as an n+<NUM> frame of the video stream recorded by video recorder <NUM>, and the process may re-progress through the storage slots <NUM>, and so forth. In some embodiments, video recorder <NUM> may be configured and/or operable to directly retrieve the images for recording from buffer <NUM>. In other embodiments, video recorder <NUM> may be configured and/or operable to indirectly retrieve the images for recording from display <NUM>. In yet other embodiment, video recorder <NUM> may be configured and/or operable to indirectly retrieve the images for recording from controller <NUM>. In yet other embodiments, video recorder <NUM> may be a part of controller <NUM>, such as a part of memory <NUM>.

In some embodiments, the algorithm may be configured and/or operable to prevent stale images from being recorded by video recorder <NUM>. Accordingly, based, at least in part, on the determination of one or more of the images in buffer <NUM> being stale, controller <NUM> may prevent the images from being recorded by video recorder <NUM>. In some such embodiments, controller <NUM> may send an error notification for recordation by video recorder <NUM>.

Embodiments of video system <NUM> may have one or more advantages. Video system <NUM> may allow for the presentation of images captured by imager <NUM> by display <NUM> when imager <NUM> and display <NUM> have different frame rates. Further, safety is greatly increased as stale video may be detected and/or prevented despite a disparity in frame rates. The disparity in frame rates between imager <NUM> and display <NUM> means that some image repetition is expected. For example, if display <NUM> has a frame rate twice that of imager <NUM>, one would expect an image captured by imager <NUM> to be presented by display <NUM> roughly twice, due to display <NUM> advancing through two frames in the time it takes a new image to be captured by imager <NUM>. This is further complicated in instances where the frame rate of imager <NUM> is variable. Thus, stale video may not be detected by the simple existence of repetition in image presentation, as false detection would commonly occur. However, the counter may allow for detecting stale video by detecting that images are repetitive in an amount greater than what may be anticipated based on the disparity in frame rates. Additionally, video system <NUM> may allow for video recorder <NUM> to record images captured by imager <NUM>, even in instances where some video recorders <NUM> may require recorded video to be recorded at a different frame rate than that provided by imager <NUM>. Further, video recorder <NUM> may record images captured by imager <NUM> at a substantially constant frame rate in contrast to a variable frame rate provided by imager <NUM>. Additionally, only storing novel hash values may have the advantage of substantially reduced memory demands. This advantage has an increasing impact so as the disparity in frame rates increases. This is such because as the disparity in frame rate increases, the number of anticipated repetitions also increases due to the faster frame advancement of display <NUM> relative imager <NUM>. These repeated images will have the same hash values. Accordingly, a smaller fraction of the assigned hashes will be stored.

As used herein, the term "and/or," when used in a list of two or more items, means that any one of the listed items can be employed by itself, or any combination of the two or more of the listed items can be employed.

The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises. a" does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

The term "substantially," and variations thereof, will be understood by persons of ordinary skill in the art as describing a feature that is equal or approximately equal to a value or description. For example, a "substantially planar" surface is intended to denote a surface that is planar or approximately planar. Moreover, "substantially" is intended to denote that two values are equal or approximately equal. If there are uses of the term which are not clear to persons of ordinary skill in the art, given the context in which it is used, "substantially" may denote values within about <NUM>% of each other, such as within about <NUM>% of each other, or within about <NUM>% of each other.

Claim 1:
A system (<NUM>) comprising:
a camera (<NUM>) configured to capture a series of images at a first frame rate;
a buffer (<NUM>) having a first number of memory storage slots (<NUM>), the buffer (<NUM>) configured to: at a frequency of time intervals, sequentially and cyclically store into the memory storage slots (<NUM>), the most current image received from the camera (<NUM>) at each interval;
a display (<NUM>) configured to present images at a second frame rate, sequentially and cyclically from the memory storage slots (<NUM>); and
a controller (<NUM>) configured to:
assign hash values to images of each frame displayed by the display (<NUM>);
store novel hash values in a memory (<NUM>);
maintain a counter, wherein:
assigning a repeat hash value increments the counter by one, and
assigning novel hash values resets the counter to zero; and
determine that one or more images in the buffer (<NUM>) are stale based, at least in part, on the counter value.