Patent Publication Number: US-11665450-B2

Title: Sensor read out mode for high resolution and low light imaging in-sync with lidar timing

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     The present application is a continuation of U.S. patent application Ser. No. 16/688,443, filed Nov. 19, 2019, the content of which is herewith incorporated by reference. 
    
    
     BACKGROUND 
     A sensor system may include several different types of sensors, such as image capture systems (e.g., cameras), radars, and/or light detection and ranging (LIDAR or Lidar) systems. Such sensor systems may be utilized, for example, in conjunction with autonomous or semi-autonomous robots and/or vehicles (e.g., self-driving cars/trucks). One challenge with these types of sensor systems is synchronizing image capture (using a camera) and lidar scans. For example, conventional systems are not designed to capture high resolution images as well as low light images using the same camera in a synchronized manner during a single lidar scan interval. 
     SUMMARY 
     The present disclosure generally relates to sensor systems and methods that provide temporally coordinated sensor information from at least two different types of sensors. 
     In a first aspect, a device is provided. The device includes an image sensor, a clock input, and a controller having at least one processor and a memory. The at least one processor is operable to execute program instructions stored in the memory so as to carry out operations. The operations include receiving, by the clock input, a clock signal. The clock signal is a periodic signal defining at least one scan interval. The operations also include during the scan interval, causing the image sensor to capture a full resolution image frame. The operations yet further include during the scan interval, causing the image sensor to capture at least one reduced resolution image frame. 
     In a second aspect, a system is provided. The system includes an image sensor, a light detection and ranging (lidar) device, and a controller having at least one processor and a memory. The at least one processor is operable to execute program instructions stored in the memory so as to carry out operations. The operations include causing the lidar device to scan a field of view based on a scan timing sequence. The scan timing sequence includes a plurality of scan intervals. The operations yet further include, during a given scan interval, causing the image sensor to capture a full resolution image frame. The operations additionally include, during the given scan interval, causing the image sensor to capture at least one reduced resolution image frame. 
     In a third aspect, a method is provided. The method includes, based on a scan timing sequence, causing a lidar device to scan a field of view. The scan timing sequence includes a plurality of scan intervals. The method also includes, during a given scan interval, causing an image sensor to capture a full resolution image frame. The full resolution image frame comprises a correlated double sampling image. The method additionally includes, during the given scan interval, causing the image sensor to capture at least one reduced resolution image frame. 
     Other aspects, embodiments, and implementations will become apparent to those of ordinary skill in the art by reading the following detailed description, with reference where appropriate to the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         FIG.  1    illustrates a schematic block representation of a device, according to an example embodiment. 
         FIG.  2    illustrates a portion of the device of  FIG.  1   , according to an example embodiment. 
         FIG.  3 A  illustrates an operating scenario, according to an example embodiment. 
         FIG.  3 B  illustrates an operating scenario, according to an example embodiment. 
         FIG.  3 C  illustrates an operating scenario, according to an example embodiment. 
         FIG.  4    illustrates a schematic block representation of a system, according to an example embodiment. 
         FIG.  5    illustrates a method, according to an example embodiment. 
         FIG.  6    illustrates an operating scenario, according to an example embodiment. 
         FIG.  7    illustrates an operating method, according to an example embodiment. 
         FIG.  8    illustrates operating scenarios, according to an example embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Example methods, devices, and systems are described herein. It should be understood that the words “example” and “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or features. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein. 
     Thus, the example embodiments described herein are not meant to be limiting. Aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein. 
     Further, unless context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should be generally viewed as component aspects of one or more overall embodiments, with the understanding that not all illustrated features are necessary for each embodiment. 
     I. Overview 
     In conventional sensor systems that include a lidar and a single camera, high resolution images camera images and low-light camera images are captured with asynchronous timing with respect to lidar scan timing. For example, in a conventional “staggered resolution readout mode” or “serial CDS image mode”, a lidar scan period could be approximately 100 milliseconds (ms). That is, a lidar could be configured to scan a predetermined region or sector of three-dimensional space during a given lidar scan period. In such conventional scenarios, a 12-megapixel image sensor/camera could be configured to capture a single 12-megapixel “high resolution” correlated double sampling (CDS) image frame followed by a 3-megapixel “low-resolution, low-light” CDS image frame. As an example, the 12-megapixel CDS image frame may have a total exposure and readout time of approximately 60-70 ms. The subsequent 3-megapixel CDS frame may have a total exposure and readout time of approximately 30 ms. However, in some cases, the subsequent low-light CDS image frame could be delayed due to readout time, integration time, and latency such that the low-light CDS image frame is complete more than 30 ms (e.g., 34 ms) after the initial lidar scan period is complete. 
     Devices, systems, and methods described herein provide various ways to temporally coordinate the various functions of high-resolution image capture, reduced-resolution image capture, and lidar scanning. For example, some embodiments may provide a high-resolution camera image that is temporally and spatially correlated with lidar-based point cloud map data, which are both obtained at a synchronized frame rate. Additionally, embodiments provide the capability to capture one or more reduced-resolution images using the same camera, all performed during a single lidar scan interval. 
     Other aspects, embodiments, and implementations will become apparent to those of ordinary skill in the art by reading the following detailed description, with reference where appropriate to the accompanying drawings. 
     II. Example Devices 
       FIG.  1    illustrates a schematic block representation of a device  100 , according to an example embodiment. Device  100  includes an image sensor  110  and a clock input  120 . In example embodiments, the image sensor  110  could include a plurality of light-sensing elements  112 . The plurality of light-sensing elements  112  are spatially grouped into a plurality of light-sensing regions  114  (e.g., a plurality of low-resolution pixels). The image sensor  110  is configured to capture the full resolution image frame  130  and a further image frame (e.g., a reduced resolution image frame  140 ) corresponding to at least one light-sensing region  114  during a single scan interval  124 . 
     In some embodiments, the image sensor  110  could include a charge-coupled device (CCD) sensor, a complementary metal-oxide-semiconductor (CMOS) sensor, and/or an active pixel sensor. It will be understood that other types of image sensors are possible and contemplated within the context of the present disclosure. 
     In some embodiments, the image sensor  110  could include more than 12 million light-sensing elements  112  (e.g., 12 megapixels, 15 megapixels, or more). 
     The device  100  also includes a controller having at least one processor  152  and a memory  154 . In some embodiments, the controller  150  may include at least one of a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). Additionally or alternatively, the at least one processors  152  may include a general-purpose processor or a special-purpose processor (e.g., digital signal processors, etc.). The processors  152  may be configured to execute computer-readable program instructions that are stored in the memory  154 . In some embodiments, the processors  152  may execute the program instructions to provide at least some of the functionality and operations described herein. 
     The memory  154  may include or take the form of one or more computer-readable storage media that may be read or accessed by the one or more processors  152 . The one or more computer-readable storage media can include volatile and/or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which may be integrated in whole or in part with at least one of the one or more processors  152 . In some embodiments, the memory  154  may be implemented using a single physical device (e.g., one optical, magnetic, organic or other memory or disc storage unit), while in other embodiments, the memory  154  can be implemented using two or more physical devices. 
     In some embodiments, the operations could include receiving, by the clock input  120 , a clock signal  122 . In such scenarios, the clock signal  122  is a periodic signal defining at least one scan interval  124 . The clock signal  122  could be an analog or digital signal that oscillates between at least a high state (e.g., +5 volts) and a low state (e.g., −5 volts). In some embodiments, the clock signal  122  could be utilized as a trigger for a synchronous digital circuit. In an example embodiment, the clock signal  122  could be generated by a clock signal generator. In some scenarios, the clock signal generator could be a part of, and/or coupled to, the device  100 . Alternatively, the clock signal generator need not be physically disposed proximate to the device  100 . 
     The operations also include, during the scan interval  124 , causing the image sensor  110  to capture a full resolution image frame  130 . In some embodiments, the full resolution image frame  130  could include a correlated double sampling (CDS) image. 
     As used herein, correlated double sampling could include a method to measure voltages or currents so as to remove an undesired signal (or lack thereof) associated with defective pixels (e.g., “dead” or “stuck” pixels). For example, when measuring the image sensor outputs, the output of the sensor could be measured twice. In such a process, the sensor output may be measured once in a “known” condition and once in an “unknown” condition. The value measured from the known condition is then subtracted from the unknown condition to generate a value with a known relation to the physical quantity being measured—in this case, photons received from an environment. 
     In some embodiments, correlated double sampling may be utilized as a noise reduction technique. For example, a reference voltage of a given image sensor pixel (i.e., the pixel&#39;s voltage after it is reset) could be subtracted from the signal voltage of the pixel (i.e., the pixel&#39;s voltage at the end of integration) at the end of each integration period. Such a subtraction may offset and/or otherwise mitigate thermal noise (e.g., kTC noise) associated with the capacitance of the respective light-sensing elements of the image sensor. 
     The operations additionally include, during the scan interval  124 , causing the image sensor  110  to capture at least one reduced resolution image frame  140 . In some embodiments, the reduced resolution image frame  140  could be a non-correlated double sampling image. That is, the reduced resolution image frame  140  could be read out from the image sensor without performing an image subtraction or normalization. 
     In some embodiments, the operations could also include receiving information indicative of a desired region of interest  160 . In such scenarios, causing the image sensor  110  to capture the at least one reduced resolution image frame  140  could include causing the image sensor  110  to capture a further image frame (e.g., second reduced resolution image frame  140   b  described in reference to  FIG.  2   ). The further image frame could correspond to the desired region of interest  160  during a given scan interval  124 . 
       FIG.  2    illustrates a portion  200  of the device  100  of  FIG.  1   , according to an example embodiment. As illustrated in  FIG.  2   , capturing the at least one reduced resolution image frame  140  could include capturing a first reduced resolution image frame  140   a  using image sensor  110 . In such scenarios, the first reduced resolution image frame  140   a  could include a first non-correlated double sampling image  142   a . In an example embodiment, the first non-correlated double sampling image  142   a  could be obtained from the same exposure as that of the full resolution image frame  130 . In such scenarios, the first reduced resolution image frame  140   a  could be provided more quickly than a CDS image because a further exposure is not needed. 
     As illustrated in  FIG.  2   , capturing the at least one reduced resolution image frame  140  could additionally include capturing a second reduced resolution image frame  140   b . In such scenarios, the second reduced resolution image frame  140   b  could include a dark frame  144 . In some embodiments, the second reduced resolution image frame  140   b  could be provided using the image sensor  110  based on a short “dark” exposure time period followed by a read out period. 
     In some embodiments, the dark frame  144  could be captured utilizing the same exposure time, ISO sensitivity, and ambient temperature as the first reduced resolution image frame  140   a . However, it will be understood that the dark frame  144  could be obtained using other exposure parameters. In some scenarios, an opaque shutter mechanism could be utilized to prevent light from reaching the image sensor  110  while capturing the dark frame  144 . 
     In some embodiments, the dark frame  144  would be utilized for sensor noise subtraction. In such scenarios, the dark frame  144  could be read out with a closed charge transfer gate TX. Accordingly, the image may be read out as a dark frame, but may include the same or similar noise as a CDS image. It will be understood that other ways to capture the dark frame  144  so as to obtain sensor noise information are possible and contemplated. 
     Furthermore, the operations performed by the controller  150  could include performing a dark image subtraction on the first reduced resolution image frame  140   a  based on the second reduced resolution image frame  140   b . In such scenarios, the dark image subtraction could correct for fixed-pattern noise such as that associated with dark current and/or “amp glow”. Visible fixed-pattern noise can be caused by “hot pixels” (e.g., pixels with higher than normal dark current), stuck pixels, and/or flickering pixels. 
     In some embodiments, the full resolution image frame  130  is formed from information received from each of the plurality of light-sensing elements  112 . For example, in cases where the image sensor  110  has 12 million light-sensing elements  112 , a corresponding full resolution image  130  may include a 12 megapixel resolution. 
     Furthermore, the reduced resolution image frame  140  is formed from information received from a subset of the plurality of light-sensing elements  112 . That is, in the case where the image sensor  110  has 12 million light-sensing elements  112 , a corresponding reduced resolution image frame  140  could include a 3 megapixel resolution. 
     In some embodiments, the scan interval  124  could be less than 40 milliseconds. For example, the scan interval  124  could correspond to a clock signal  122  (e.g., a lidar clock signal) with a period of about 30 milliseconds. However, it will be understood that longer or shorter scan intervals are possible and contemplated. 
     In various examples, the clock input  120  could be based on a scan timing sequence  430  of a light detection and ranging (lidar) device (e.g., lidar device  410 ) as illustrated and described in reference to  FIG.  4   . 
     In such scenarios, the full resolution image frame  130  could be captured while the lidar device is scanning a field of view (e.g., field of view  420 ). Additionally or alternatively, the at least one reduced resolution image frame  140  can be captured while the lidar device is not scanning the field of view. 
       FIG.  3 A  illustrates an operating scenario  300 , according to an example embodiment. Operating scenario  300  could illustrate a “serial image capture” scenario. As an example, to provide a first full resolution CDS image frame  306   a , during scan interval  124   a  (e.g., between to and ti), the image sensor  110  could be exposed to light and accumulate charge during exposure  302   a  and the accumulated charge could be read out during read out  304   a . Furthermore, to provide a first reduced resolution CDS image frame  308   a , the image sensor  110  could again be exposed to light and accumulate charge during exposure  302   b  and the corresponding accumulated charge could be read out during read out  304   b.    
     The serial image capture process illustrated in operating scenario  300  could continue during scan intervals  124   a ,  124   b , and  124   c . For example, a second full resolution image frame  306   b  could be captured over exposure  302   c  and read out  304   c  and a second reduced resolution image frame  308   b  could be captured over exposure  302   d  and read out  304   d . However, the serial capture process may result in image frames that are not synced with the scan intervals  124   a - c  or clock signal  122 . Accordingly, the information about objects and other features in the image frames captured in operating scenario  300  may be more difficult to incorporate and/or compare to other types of information captured about the environment obtained based on the scan intervals  124   a - c . For example, in the case of a lidar device, image sensor data captured in a serial capture process could be spatially offset with respect to lidar data and/or more difficult to utilize for sensor fusion and/or other perception determinations. 
       FIG.  3 B  illustrates an operating scenario  320 , according to an example embodiment. Operating scenario  320  could include, during scan interval  124   a , capturing a first full resolution image frame  326   a  during exposure  322   a  and read-out  324   a . Subsequently, a first reduced resolution non-CDS image frame  328   a  could be captured during read-out  324   b . The first reduced resolution non-CDS image frame  328   a  could utilize the same exposure  322   a  as the first full resolution image frame  326   a . However, due to the non-CDS property of the first reduced resolution image frame  328   a , the resulting image could be noisier or otherwise of less quality than an equivalent CDS reduced resolution image frame. 
     During subsequent scan intervals (e.g., scan intervals  124   b  and  124   c ), a second full resolution image frame  326   b  could be captured during exposure  322   b  and read-out  324   c  and a second reduced resolution image frame  328   b  could be captured during read-out  324   d . Although only partially illustrated, a third full resolution image frame  326   c  could be captured during exposure  322   c  and a corresponding read-out time. 
     By operating according to operating scenario  320 , the device  100  could provide full and reduced resolution image frames that are in synchronization with the scan intervals  124   a - c . By synchronizing with the scan intervals  124   a - c , sensor fusion and/or perception tasks could be made more efficient and less computationally-intensive. 
       FIG.  3 C  illustrates an operating scenario  330 , according to an example embodiment. Operating scenario  330  includes capturing a first full resolution image frame  336   a  corresponding to exposure  332   a  and read-out  334   a . Afterwards, the operating scenario  330  includes capturing a first reduced resolution image frame  338   a  during read-out  334   b . Next, the operating scenario  330  includes capturing a dark frame  340   a  associated with exposure  332   b  and read-out  334   c . In operating scenario  330 , the first reduced resolution image frame  338   a  could be non-CDS, while the dark frame  340   a  could be an image obtained with a closed charge transfer gate (e.g., a “closed” electronic shutter) or with a closed physical shutter. 
     In subsequent scan intervals  124   b  and  124   c , the capture sequence could repeat with capturing a second full resolution image frame  336   b , a second reduced resolution image frame  338   b , and a second dark frame  340   b.    
     By utilizing such an operating mode, the device  100  could be configured to provide a full resolution image frame and a dark-current-corrected reduced resolution image frame during each scan interval. 
     III. Example Systems 
       FIG.  4    illustrates a schematic block representation of a system  400 , according to an example embodiment. System  400  could include elements that may be similar or identical to that of device  100 , illustrated and described in relation to  FIG.  1   . For example, system  400  includes an image sensor  110 . 
     System  400  also includes a light detection and ranging (lidar) device  410 . Lidar device  410  could be configured to provide information (e.g., point cloud data) about one or more objects (e.g., location, shape, etc.) in a given environment. In an example embodiment, the lidar system could provide point cloud information, object information, mapping information, or other information to a vehicle. The vehicle could be a semi- or fully-automated vehicle. For instance, the vehicle could be a self-driving car, an autonomous drone aircraft, an autonomous truck, or an autonomous robot. Other types of vehicles and LIDAR systems are contemplated herein. 
     Furthermore, system  400  also includes a controller  150  that includes at least one processor  152  and a memory  154 . The at least one processor  152  is operable to execute program instructions stored in the memory  154  so as to carry out operations. 
     In some embodiments, the operations could include causing the lidar device  410  to scan a field of view  420  based on a scan timing sequence  430 . In such scenarios, the scan timing sequence  430  could include a plurality of scan intervals  124 . The operations may also include, during a given scan interval  124 , causing the image sensor  110  to capture a full resolution image frame  130 . The operations may additionally include, during the given scan interval  124 , causing the image sensor  110  to capture at least one reduced resolution image frame  140 . In some embodiments, the full resolution image frame  130  could be captured while the lidar device  410  is scanning the field of view  420 . Additionally or alternatively, the at least one reduced resolution image frame  140  could be captured while the lidar device  410  is not scanning the field of view  420 . 
     As illustrated in  FIG.  4   , in some embodiments, the controller  150  could transmit lidar control signals  158  to the lidar device  410 . The lidar control signals  158  could be used so as to maintain and/or change and operation of the lidar device  410 . 
     In some embodiments, the system  400  could be controlled according to operating scenarios  300 ,  320 , and/or  330 , corresponding to  FIGS.  3 A,  3 B, and  3 C . 
     IV. Example Methods 
       FIG.  5    illustrates a method  500 , according to an example embodiment. It will be understood that the method  500  may include fewer or more steps or blocks than those expressly illustrated or otherwise disclosed herein. Furthermore, respective steps or blocks of method  500  may be performed in any order and each step or block may be performed one or more times. In some embodiments, some or all of the blocks or steps of method  500  may relate to elements of device  100  and/or system  400  as illustrated and described in relation to  FIGS.  1  and  4   . Some steps or blocks of method  500  could be illustrated and described in relation to  FIGS.  3 B- 3 C,  6 , and  7   . 
     Block  502  includes, based on a scan timing sequence (e.g., scan timing sequence  430 ), causing a lidar device (e.g., lidar device  410 ) to scan a field of view (e.g., field of view  420 ). The scan timing sequence includes a plurality of scan intervals (e.g., scan interval(s)  124 ). 
     Block  504  includes, during a given scan interval, causing an image sensor (e.g., image sensor  110 ) to capture a full resolution image frame (e.g., full resolution image frame  130 ). In such scenarios, the full resolution image frame could include a correlated double sampling image. 
     Block  506  includes, during the given scan interval, causing the image sensor to capture at least one reduced resolution image frame. In such scenarios, capturing the at least one reduced resolution image frame could include capturing a first reduced resolution image frame (e.g., first reduced resolution image frame  338   a ). In some examples, the first reduced resolution image frame could include a non-correlated double sampling image. Furthermore, capturing the at least one reduced resolution image frame could also include capturing a second reduced resolution image frame (e.g., second reduced resolution image frame  340   a ). In some embodiments, the second reduced resolution image frame could include a dark frame. The method  500  also includes performing a dark image subtraction on the first reduced resolution image frame based on the second reduced resolution image frame. 
     In some embodiments, the full resolution image frame could be captured while the lidar device is scanning the field of view. As an example, the at least one reduced resolution image frame could be captured while the lidar device need not be scanning the field of view. 
     In some embodiments, the given scan interval could be less than 40 milliseconds (e.g., 20-30 ms). However, other scan intervals are possible and contemplated. 
       FIG.  6    illustrates an operating scenario  600 , according to an example embodiment. Operating scenario  600  includes an image sensor  610 , a smart sensor  620 , and a central controller  630 . The operating scenario  600  could include obtaining information (e.g., pixel charge amount) by using an image sensor  610 . Thereafter, operating scenario  600  may include processing at least a portion of the obtained information using smart sensor  620 . The smart sensor  620  could include one or more circuits configured to efficiently transmit and/or filter information provided by the image sensor  610  based on, for example, a desired field of view, before passing it along to the central controller  630 . In turn, the central controller  630  could be to process the information and/or present the processed information to a user. 
       FIG.  7    illustrates an operating scenario  700 , according to an example embodiment. In some embodiments, the operating scenario  700  could include an image sensor  610  capturing a high resolution image. Next, the image sensor  610  could capture a low resolution non-CDS image. Subsequently, the image sensor  610  could capture a low-resolution dark frame. As described herein, the dark frame could be captured with a closed charge transfer gate, TX. Such an image may include similar or identical noise information as a CDS image, providing a noise reference frame that may be subtracted from the low resolution non-CDS image. 
     In some embodiments, the smart sensor  620  could be configured to perform a dark image subtraction between the two low resolution images in an effort to reduce noise due to malfunctioning pixels and/or related read out circuitry. Thereafter, the smart sensor  620  could be configured to further adjust or finalize the image and send a processed frame to the central controller  630 . After the processed frame is transmitted, this method could repeat again during a subsequent scan interval. 
       FIG.  8    illustrates operating scenarios  800  and  810 , according to an example embodiment. Operating scenario  800  illustrates a serial image capture process similar or identical to that described in reference to  FIG.  3 A . In the serial image capture process, a CDS full resolution image frame is captured immediately followed by a CDS reduced resolution image frame. However, due to the length of time needed to do back-to-back CDS image frames, after the first full resolution image frame, subsequent image frames can be “out-of-sync” with respect to lidar scan intervals. 
     Operating scenario  810  illustrates a process similar or identical to that described in reference to  FIG.  3 C . In such embodiments, a CDS full resolution image frame (e.g., 12 megapixel resolution) could be captured, triggered by a rising or falling edge of a scan trigger pulse (e.g., trigger pulse  126 ). Afterward, a non-CDS reduced resolution image frame (e.g., 3 megapixel resolution) could be captured, followed by a reduced resolution dark frame. In such a manner, a CDS full resolution image frame and a non-CDS, dark-image-subtracted, reduced resolution image frame could be provided “in sync” with the lidar scan intervals. 
     It will be understood that while the respective image frames are captured in a particular order, other orders of image frame capture are contemplated and possible. 
     The particular arrangements shown in the Figures should not be viewed as limiting. It should be understood that other embodiments may include more or less of each element shown in a given Figure. Further, some of the illustrated elements may be combined or omitted. Yet further, an illustrative embodiment may include elements that are not illustrated in the Figures. 
     A step or block that represents a processing of information can correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a step or block that represents a processing of information can correspond to a module, a segment, a physical computer (e.g., a field programmable gate array (FPGA) or application-specific integrated circuit (ASIC)), or a portion of program code (including related data). The program code can include one or more instructions executable by a processor for implementing specific logical functions or actions in the method or technique. The program code and/or related data can be stored on any type of computer readable medium such as a storage device including a disk, hard drive, or other storage medium. 
     The computer readable medium can also include non-transitory computer readable media such as computer-readable media that store data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer readable media can also include non-transitory computer readable media that store program code and/or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, compact-disc read only memory (CD-ROM), for example. The computer readable media can also be any other volatile or non-volatile storage systems. A computer readable medium can be considered a computer readable storage medium, for example, or a tangible storage device. 
     While various examples and embodiments have been disclosed, other examples and embodiments will be apparent to those skilled in the art. The various disclosed examples and embodiments are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.