Quality of optically black reference pixels in CMOS iSoCs

Aspects relate to improved optically black reference pixels in a CMOS iSoc sensor. A system can include a pointer P1 that indicates pixels to be read out during a readout time interval, a pointer P2 that indicates pixels to be reset during the time interval, and a pointer P3 that preserves a validity of a frame. The system also includes a pointer P4 configured to mitigate an integration time of column fixed pattern noise (FPN) rows independently of the integration time of other rows. In some aspects, pointer P4 can mitigate blooming into sampled rows from surrounding rows. Pointer P4 can be continuously rotated, in an aspect. Further, in some aspects, pointer P4 can jump on a second cycle to arrive one line before pointer P1.

BACKGROUND

Technological advances have led to complementary metal-oxide-semiconductor (CMOS) sensor images being leveraged for use in digital cameras, camcorders, video systems, and the like. CMOS is low cost and versatile and, thus, has become the technology of choice for many image sensor arrays. Within CMOS itself, many types of devices intended for visible imaging applications are in use. Such devices can be tailored to large-format still cameras, standard video cameras, and compact “web cam” units, for example, all with varying degrees of size, cost, and performance.

CMOS sensor images can include an integrated circuit with an array of pixel sensors, each of which can comprise a photodetector. Further, a CMOS sensor imager can be incorporated into a System-on-Chip (SoC), which can integrate various components (e.g., analog, digital, and so forth) associated with imaging into a common integrated circuit. For example, the SoC can include a microprocessor, microcontroller, or digital signal processor (DSP) core, memory, analog interfaces (e.g., analog to digital converter, digital to analog converters), and so forth.

Visible imaging systems implemented using CMOS imaging sensors can reduce costs, power consumption, and noise while improving resolution. For example, cameras can use CMOS imaging System-on-Chip (iSoC) sensors that efficiently merge low-noise image detection and signal processing with multiple supporting blocks that can provide timing control, clock drivers, reference voltages, analog to digital conversion, digital to analog conversion, and key signal processing elements. High-performance video cameras can thereby be assembled using a single CMOS integrated circuit supported by few components including a lens and a battery, for example. Accordingly, by leveraging iSoC sensors, camera size can be decreased and battery life can be increased. Also, dual-use cameras have emerged that can employ iSoC sensors to alternately produce high-resolution still images or high definition (HD) video.

A CMOS imaging sensor can include an array of pixel cells, where each pixel cell in the array can include a photodetector (e.g., photogate, photoconductor, photodiode, and so on) that overlays a substrate for yielding a photo-generated charge. A readout circuit can be provided for each pixel cell and can include at least a source follower transistor. The pixel cell can also include a floating diffusion region connected to a gate of the source follower transistor. Accordingly, charge generated by the photodetector can be sent to the floating diffusion region. Further, the imaging sensor can include a transistor for transferring charge from the photodetector to the floating diffusion region and another transistor for resetting the floating diffusion region to a predetermined voltage level prior to charge transference. A floating diffusion region of a pixel cell is commonly reset by opening a circuit to a reset voltage source. Such opening of the circuit can be managed by digital control.

A typical CMOS sensor records images on a frame by frame basis; the amount of light integrated during a particular frame is linearly dependent on the duration of each frame. Additionally, the duration of each frame is inversely related to the sensor frame rate such that faster frame rates allow less light to be integrated into each pixel. Various light integration modes can be employed by a CMOS imaging sensor. For instance, in full frame integration mode, each pixel can be integrated or exposed to a light source at almost any time during the duration of a full frame time except when the pixel is being read and reset. This mode can allow for the maximum amount of light to be integrated in each pixel, which can provide high signal integration. Further, in sub-frame integration mode, each pixel can be integrated or exposed to a light source for a period of time, which is less than a full frame time while maintaining the same frame rate as for the full frame integration mode.

SUMMARY

An aspect relates to a system for improved optically black reference pixels in a complementary metal-oxide-semiconductor (CMOS) imaging system-on-chip (iSoC) sensor. The system includes a read component configured to indicate pixels to be read out during a readout time interval. The system also includes a first reset component configured to indicate pixels to be reset during the readout time interval and a second reset component configured to preserve a validity of a frame during the readout time interval. Further, the system includes a control component configured to facilitate an integration time of column fixed pattern noise (FPN) rows independently of an integration time of other rows. Additionally, the control component can be configured to mitigate the integration time of the column FPN rows as compared to the integration time of the other rows.

Another aspect relates to a method for improved optically black reference pixels in a complementary metal-oxide-semiconductor (CMOS) imaging system-on-chip (iSoC) sensor. The method includes placing a first pointer in idle mode when column fixed pixel noise (FPN) optical black (OB) rows are not being read. The method also includes implementing the first pointer one line before a second pointer arrives at a first column FPN OB row. The method further includes diverting a third pointer and a fourth pointer away from the column FPN OB rows when the third pointer and the fourth pointer reach the column FPN OB rows.

Still another aspect relates to a method that includes indicating pixels to be read out and reset during a read out time interval and preserving a validity of a frame. The method also includes facilitating during the full valid frame an integration time of column fixed pattern noise (FPN) rows independently of an integration time of other rows through utilization of a vertical pointer. In an implementation, the method includes mitigating against the occurrence of extraneous signal(s) blooming into sampled rows from surrounding rows with the vertical pointer.

DETAILED DESCRIPTION

Conventional complementary metal-oxide-semiconductor (CMOS) imaging Sensors-on-Chip (iSoC) can be prone to revealing undesirable artifacts (e.g., rolling shutter skew, partial exposure, and so forth) arising from some beneficial readout architectures that exist on the CMOS iSoC for reading the output images. The basis of optical image detection in a CMOS iSoC is charge accumulated within an optically sensitive element known as a photodiode. Since it can be efficient to integrate only one signal storage site at each pixel, specifically the photodiode and ancillary capacitances, the image typically is formed on a row-by-row basis rather than simultaneously across the entire image sensor.

The photodiode is a floating, reverse-biased silicon p-n junction diode. A photon incident upon the depletion zone of such a diode may, depending upon its wavelength and probabilistic factors, be capable of promoting an electron from the valance band to the conduction band, leaving behind a hole. For example, the reverse-biased diode can have a depletion zone around the junction and, when a photon is incident on that depletion zone, the photon can have a certain probability to convert an electron, which gives rise to an optical image. The resultant electrons and holes can drift along the electric field lines created by the reverse-bias potential. The drifting of the electrons and holes can occur until collected by the conducting electrode of the opposite sign.

The total number of accumulated electrons within a defined unit of time can serve as a proportional analog for the number of photons incident upon the volume of silicon occupied by the photodiode's depletion zone. The collection of electron-hole pairs continues to a limit, which occurs when the floating diode is neutralized. The point at which the floating diode is neutralized is referred to as the “full-well”.

An idealized sensor is a sensor in which the charge information is immediately (or as quickly as possible) converted to digital information. Once information is stored in the digital domain, the potential for corruption is minimized (e.g., rendered almost non-corruptible). In reality, demands for high spatial resolution, high sensitivity (fill factor) and small optical formats drive the technology towards minimal functionality and maximum areal efficiency within the pixel.

Megapixel CMOS imaging arrays typically have only around three to around five transistors per pixel. Therefore, the accumulated charge information should be relayed to remote digitizers. This can be accomplished with small numbers of high speed analog-to-digital converters (ADCs) and an elaborate analog fan-in (e.g., used to add or combine multiple analog signals and ORs logic signals for later processing). The charge can be converted to a voltage at the pixel and the voltage information can be transmitted through several buffering/amplification stages.

Such analog readout schemes tend to introduce undesirable artifacts within the output image. These artifacts are especially noticeable in low-light/high-gain scenarios in which a large proportion of the dynamic range is supplied to the read noise. In video applications, the most aesthetically pleasing dark image is one in which only uniform (e.g., nearly flawless) Gaussian pixel temporal noise is visible. Any additional visible noise types are undesirable. In particular, the presence of any fixed pattern noise (FPN) can be undesirable. For example, anything that is present in the image that does not change frame-to-frame is not desired.

One particularly egregious artifact is known as “column FPN”. This appears primarily as a consequence of having an independent analog buffer within each column. Each buffer may introduce a small, random offset resulting in visible vertical stripes within a dark image.

A purely digital correction scheme can be employed to address column FPN. For example, a subset of optically black pixels can be used to measure the voltage offset of each column. The optically black pixels can be on the top of every column to indicate the offset of the particular column. These column offset values can be stored in RAM (Random Access Memory) or another type of storage media. Further, the column offset values can be subtracted from the data from the optically sensitive (clear) portion of the pixel array.

The efficacy of such a correction may be limited by many factors including the quality of the small available sample of optical black (OB) pixels per column and their fidelity with respect to the clear pixels. In particular, the most significant quadratic term describing the performance limit is proportional to the magnitude of pixel FPN within the OB pixels. For example, the number of OBs available is a direct statistical term that depends on the square root of the number of OBs available in terms of the precision of the column offset computation. Thus, the most significant quadratic term depends on the pixel FPN, which is the pixel to pixel offset standard deviation divided by the square root of optical black pixels in the column.

The pixel FPN has two components which sum in quadrature. The two components are (1) a fixed component (minimal pixel FPN) and (2) a component which scales linearly with integration time (the DSNU (Dark Signal Non-Uniformity) term). DSNU can be caused by (and be mathematically equivalent to) the pixel-wise dispersion of dark current. The DSNU might dominate the pixel FPN, particularly at elevated temperatures and at large integration times.

Another limiting factor arises from the number of available OB pixels. The more OBs available, the lower the statistical error in the column offset estimate. It is thus desirable in video applications to use all of the OB information for which there is the time to sample. This, however, might be limited by the available vertical blanking period which is dictated by the SMPTE (Society of Motion Picture and Television Engineers) video standards. For example the high definition 1080P standard has 1125 line periods and 1080 image lines per frame. Therefore, the maximum OB pixels that could theoretically be sampled is 45 per column.

In a regular rolling shutter application dictated by efficient pixel design, wherein the accumulated signal is stored directly on the photodiode and assorted capacitances, an array is scanned one row at a time for readout. Thus, the pixels are not exposed at the same time but are reset in sequence line-by-line and read out in sequence, line-by-line. Consequently, in such a rolling shutter application, all of the sampled OBs cannot be utilized for correction. For example, the OBs at the edges of the sampled window are adjacent to pixels which are not routinely reset since they are not visited by the rolling (vertical) pointers. These non-reset pixels overflow with charge, which may spill into neighboring pixels, known as “blooming”. For example, in blooming, the pixels are saturated with dark current and any charge that is generated beyond the saturation limit can overwhelm the storage site and migrate to neighboring pixels. Therefore, edge pixels tend not to have high fidelity since they can often become corrupted by charge spilling into them from their blooming neighbors.

To address the problem of blooming, the disclosed aspects provide a vertical pointer that addresses the limitation of DSNU within optically black pixels. Further, the vertical pointer can provide for a means to use all (or substantially all) of the available OB pixel statistics by making the sampled OBs immune to blooming. Further, the disclosed aspects can provide an independent exposure time in the optically black pixel being used for the correction. In accordance with some aspects, the disclosed aspects can improve quality of optically black reference pixels by enabling shortening of the integration time relative to the active pixel integration time. The exposure time of the optically black pixel is independent of what is occurring in the remainder of the array, according to an aspect. As used herein, the vertical pointer might alternatively be referred to as an “additional pointer”, “additional vertical pointer”, “first pointer” or “P4”.

Herein, an overview of some of the embodiments for improving the quality of optically black reference pixels in CMOS iSoCs has been presented above. As a roadmap for what follows next, various exemplary, non-limiting embodiments and features for improving the quality of optically black reference pixels in CMOS iSoCs are described in more detail. Then, some non-limiting implementations and examples are given for additional illustration, followed by a representative operating environment in which such embodiments and/or features can be implemented.

With reference toFIG. 1, illustrated is a system100that generates digital signals by employing a CMOS sensor imager. The system100can be associated with a CMOS sensor imager utilized in connection with a camcorder, digital camera, microscope, video system, and/or the like. The system100comprises a pixel array102that can include M rows and N columns of pixel cells, where M and N can be any integers. Each pixel in the pixel array102can comprise a photodetector (e.g., photogate, photoconductor, photodiode, and so forth). Further, each pixel in the pixel array102can be utilized to detect a particular color of light; thus, a subset of the pixels in the pixel array102can operate in response to red light (R pixels), a disparate subset of the pixels can operate based upon blue light (B pixels) and a further subset of the pixels can operate as a function of green light (G pixels). Other color filter combination can also be used with the so-called Bayer construction most dominant.

The pixel array102can include a portion104covered by metal. The metal covered portion104can include optical black pixels; thus, pixels included in the metal covered portion104lack exposure to light due to being covered by metal sufficiently opaque to transmission of light, as discussed in U.S. Pat. No. 7,999,340, entitled “Apparatus and method for forming optical black pixels with uniformly low dark current”, which is incorporated by reference herein. Any number of rows (out of the M rows) and/or columns (out of the N columns) can be included in the metal covered portion104of the pixel array102. The pixel array102can additionally include a clear pixel portion106. Pixels in the clear pixel portion106can be exposed to light. It is to be appreciated that any number of rows and/or columns can be included in the clear pixel portion106. Moreover, the metal covered portion104can, but need not, be symmetrically located upon the pixel array102with respect to the clear pixel portion106. For example, a first number of rows can be covered by metal at a top of the pixel array102and a second number of rows can be covered by metal at a bottom of the pixel array102such that the first and second numbers can be the same or different.

An image focused on the pixel array102(e.g., the clear pixel portion106) can cause the pixels to convert incident light into electrical energy. Signals obtained by the pixel array102can be processed on a column by column basis; thus, a particular row of pixels from the pixel array102can be selected to be read. The system100can further include a plurality of read buses108that can transfer the contents from the pixels in the pixel array102in the selected row. According to an illustration, the system100can include Q read buses108, where each read bus108can be associated with a respective column of the pixel array102and where Q is an integer. By way of further example, pixels in the pixel array102can share read buses108, and thus, the system100can include fewer than Q read buses108.

Each read bus108can carry content (e.g., sampled signals) from the pixels to a respective column buffer (CB)110. The system100can include R column buffers110or fewer, for instance, where R is an integer. The column buffers110can amplify (e.g., condition) the signals from the pixels. Further, each column buffer110can enable low noise readout and can condition the signal from a pixel positioned at one of the rows in the column (or columns) corresponding to the column buffer110.

After processing by the column buffers110, outputted values from each of the column buffers110can be retained. Moreover, each of the column buffers110can be associated with respective circuitry such as, for instance, a respective capacitor112and switch114. Such circuitry can sample and hold outputted values from the corresponding column buffers110. For example, the capacitors112can be loaded with the outputted values from the corresponding column buffers110. Further, the switches114can be closed one at a time to allow for connecting to a bus116; thus, the voltages generated by the column buffers110can be multiplexed over the bus116. The bus116can enable communicating each of the outputted values from the respective column buffers110to an analog to digital converter (ADC)118. The ADC118can digitize the sampled signal to yield a digital signal120. The digital signal can thereafter be provided to disparate component(s) (not shown) for further processing, manipulation, storage, and so forth.

Images yielded from the digital signal120can be negatively impacted by fixed pattern noise (FPN). Fixed pattern noise can be introduced into the signal based upon processing effectuated within the system100(e.g., by the pixels of the pixel array102, column buffers110, ADC118, and so forth). For example, different pixels can yield disparate, respective fixed pattern noise, differing column buffers110can generate differing, respective fixed pattern noise, and so forth (e.g., due to mismatch between the pixels, column buffers110, and so on). Accordingly, calibration techniques can be employed to mitigate the fixed pattern noise.

The optical black pixels in the metal covered portion104of the pixel array102can be employed for calibration by providing references from which noise levels can be deduced. By way of illustration, a row of optical black pixels in the metal covered portion104can be selected to be read. Since this row of optical black pixels fail to receive light, signals yielded by each of these pixels lack correlation to incident light upon the pixel array102(e.g., zero input is provided to the optical black pixels). The signals generated by the optical black pixels can be processed in a similar manner as compared to pixels from the clear pixel portion106of the pixel array102(e.g., pixels in the same column can be read through the same column buffer110). Since zero input is provided to the optical black pixels, it can be expected that zero output should be yielded upon processing; however, noise can be included in the outputted signals.

Accordingly, calibration can be effectuated to determine the noise associated with each column, which can be referred to as the column fixed pattern noise (column FPN). Noise values associated with each column of the pixel array102can be determined during calibration. For example, digital signals corresponding to the row of optical black pixels can be retained in a line of memory, where each of the digital signals can correlate to noise of a particular column. Thereafter, the set of noise values from the line of memory can be utilized during a correction phase to mitigate column FPN within the outputted digital signal (e.g., subtract a noise value associated with a particular column from a signal value of a clear pixel from the particular column).

According to some aspects, digital signals from any number of rows of optical black pixels can be combined in any manner. For example, various calibration algorithms can be utilized by the system100(e.g., determining average, median, mode, and so forth, of digital signals from optical black pixels in each column over time, aging out older values of digital signals from optical black pixels, and so on).

In accordance with some aspects, each analog readout element (e.g., column buffer, line-driver, analog PGA (Programmable Gain Amplifier) and ADC) can introduce a finite offset deviation. The net combinatorial effect of the finite offset deviations, results in column FPN. In some aspects, a pure digital correction is utilized to render the column offsets uniform.

The individual (signed) offset of each column can be determined using representative optically black (OB) pixels, which can be stored in memory. When the clear pixels are being read, the offsets can be accordingly subtracted. Assuming all variations are Gaussian, the “post-correction” column FPN, σcdepends on: the pixel FPN, σP, within the OB pixels and the number of available OB pixels per column nOB
σc=σP/√{square root over (n)}OB

In order for the column FPN correction to be effective, in post correction the resulting column FPN should be less than one-tenth (< 1/10) of the pixel noise in still capture applications and less than one-twentieth (< 1/20) of the pixel read noise in video applications at 60 Hz frame rate.

Since the column FPN depends linearly on the pixel FPN within the OB pixels, every possible measure should be taken to minimize the pixel FPN. The pixel FPN, σP, is normally dominated by dark current dispersion (DSNU) especially at realistic operating temperatures and significant integration times.
σP=√(m2+t2·D2)
where m is the minimal pixel FPN, t is the integration time, and D is the DSNU.

In accordance with some aspects as illustrated by the example system200ofFIG. 2, vertical reset pointers (P1, P2, P3, and P4) are provided to operate the OB rows that are used for the column FPN correction independently from the other rows. For example, the OB rows for column FPN correction can have an independent integration time with respect to the other rows. The independent integration time can be tuned to and held at its minimum (e.g., 1 line).

The example system200includes a pixel array102, which comprises a plurality of pixels, where four pixels are represented by the blocks in the pixel array102. The system200can include a read component202that is configured to determine the line that is currently being read out. In an aspect, read component202can be configured to indicate pixels to be read out during a readout time interval In some aspects, read component202can be configured to control pointer P1.

Also included in system is a first reset component204configured to indicate pixels to be reset during the readout time interval. In an example, first reset component204is configured to effect a second reset phase for each row, which provides tunable sub-frame integration times. In an aspect, first reset component204is configured to control pointer P2.

System200also includes a second reset component206configured to preserve a validity of a frame. For example, the second reset component206can be configured to phase pointer P3in and out of existence when changes are made to a vertical window or an integration time. The transient nature of pointer P3can be utilized to preserve the validity of the frame which follows such changes.

The system200also includes a control component208configured to facilitate an integration time of column fixed pattern noise (FPN) rows independently of the integration time of other rows. In some aspects, control component208can be configured to mitigate the integration time of FPN rows. For example, control component208can be configured to retain the integration time of the column FPN rows to the minimal value, independent of what the other rows are doing. Such independent integration time management can facilitate minimizing the pixel FPN, according to an aspect.

Depending on the implementation details, the control component208can also be configured to provide a means of mitigating blooming into the sampled rows from the surrounding rows. For example, the control component208can be configured to mitigate blooming into the OB rows from their neighbors which previously were not reset with an implementation that utilizes the read component202and the first reset component204. In another example, the control component208can be configured to mitigate blooming into the first clear row and the last clear row, which otherwise also has neighboring non-reset rows.

In accordance with some aspects, control component208can be configured to manage pointer P4. For example, control component208can be configured to place pointer P4in an idle mode when column FPN OB rows are not being read out and can implement pointer P4one line before read component202(e.g., pointer P1) arrives at a first column FPN OB row. Further to this aspect, first reset component204and second reset component206can be configured to divert pointers P2and P3, respectively, away from the column FPN OB rows when pointers P2and P3reach the column FPN OB rows.

In accordance with some aspects, control component208can be configured to allocate a first subset of dummy rows to pointer P4and first reset component204can be configured to allocate a second subset of dummy rows to pointer P2. The first set of dummy rows can be independent of the second subset of dummy rows. Further to this aspect, control component208can be configured to send pointer P4to the first subset of dummy rows when the column FPN OB rows are not being read.

In accordance with some aspects, control component208is configured to continuously rotate pointer P4within the column FPN OB rows. According to some aspects, control component208is configured to jump pointer P4on the second cycle such that pointer P4arrives at the first FPN OB row one line before pointer P1arrives at the first FPN OB row.

According to some aspects, control component208is configured to extend an activity region of pointer P4to rows that are not read. Further to this aspect, control component208can cycle pointer P4within the rows that are not read and can jump pointer P4to a start of the FPN OB window one line ahead of pointer P1.

In other aspects, control component208can be configured to use pointer P4to reset a set of extra rows around an active window. Further to this aspect, control component208can be configured to synchronize pointer P4to a frame.

To provide further details related to the disclosed aspects,FIG. 3illustrates a first example implementation300, according to an aspect. For purposes of explanation, the simplistic case of zero vertical blanking is considered. In the figure, a simplified view of the array is considered. For example, the dummy OBs and the bottom OB rows are not illustrated. Further, there are only OBs at the top of the array in this simplified view. P2is assumed to be active in the dummy rows when it is not present (e.g., disappears) inFIG. 3.

Time is illustrated along the horizontal axis. The OB Clamp OB Rows are indicated at302, the Column FPN Correction OB Rows are indicated at304, and Clear Window Rows are indicated at306. Unread rows are indicated at308,310,312, and314. Further, P1is indicated by line316, P2is indicated by line318, and P4is indicated by line320.

FIG. 3illustrates an implementation wherein the fourth pointer (P4) is idle when the column FPN OB rows304are not being read. In this case, pointer P4320is transient and comes into existence one line (in time) before pointer P1316arrives at the first column FPN OB row304. This is indicated, in one example instance, by dashed oval322. Pointer P4320, arriving one line (in time) before pointer P1316, provides one line of integration. Pointer P4320disappears one line before the last column FPN row304is read, as indicated in one example instance by dashed oval324.

Pointer P2318and P3(not shown for purposes of simplicity) do not visit the column FPN OB rows304. Instead pointer P2318and P3are diverted to the dummy rows when they reach the first column FPN OB row, as indicated by dashed ovals326and328, in two example instances.

The example implementation ofFIG. 3does not provided for anti-blooming protection, which can be acceptable in various implementations. Since pointer P4320is only active for a small fraction of the frame time (e.g., during the FPN OB row), it can introduce an imbalance in the activity levels throughout the frame. For example, in sub-frame integration mode, the environment seen by pointer P2318can change while pointer P2318is occupied resetting rows which are mid-frame. This could potentially lead to a visible bar in the image. To mitigate this effect, pointer P4320can be sent to a subset of dummy rows when it is not needed. To implement sending pointer P4320to the dummy rows, a set of dummy rows can be allocated to pointer P4. To mitigate collisions, the set of dummy rows allocated to pointer P4are independent of a set of dummy rows allocated to pointer P2318. Thus, according to the aspects that utilize pointer P4without blooming protection, physical rows served by pointer P4are not visited by pointer P2nor pointer P3and physical rows served by pointer P2and/or pointer P3are not visited by pointer P4, which can mitigate pointer collision.

FIG. 4illustrates a second exemplary implementation400that utilizes a continuously rotating pointer P4, according to an aspect. For purposes of explanation, the simplistic case of zero vertical blanking is considered and the dummy OBs are not shown. Further, there are only OBs at the top of the array for purposes of simplicity.

An advantage of the option to continuously rotate pointer P4320is that the environment seen by pointer P2318can be uniform throughout the frame, which should mitigate the possibility of viewing a bar in the image, as discussed above with reference toFIG. 3. If the frame period is not an exact multiple of the number of column FPN OB rows304, the entry point (within the column FPN OB rows304) for pointer P1316, which provides a single line of integration, can vary frame to frame.

As illustrated by oval402, on the second cycle, pointer P1316is entering the FPN OB rows304midway through the window. In order to complete its requisite cycle of OBs, pointer P1316can jump to the start of the window when it reaches the end and then exit mid-window.

FIG. 5illustrates a third exemplary implementation500that utilizes a jumping pointer P4, according to an aspect. For purposes of explanation, the simplistic case of zero vertical blanking is considered and the dummy OBs are not shown. Also, there are only OBs at the top of the array for purposes of simplicity.

An alternative to having a variable pointer P1316entry point is to have the first row within the window read first and to charge the pointer P4320with the function of jumping in order to maintain a single line of integration. This can simplify the implementation of the pointer P1316at the expense of the pointer P4320. Starting with a different physical FPN OB each frame should not compromise the FPN correction.

As indicated by the oval502, pointer P4320is observed to jump (e.g. on the second cycle) in order to arrive the first FPN OB row one line before pointer P1316arrives.

FIG. 6illustrates a fourth exemplary implementation600that incorporates anti-blooming features, according to an aspect. For purposes of explanation, the simplistic case of zero vertical blanking is considered and the dummy OBs are not shown. Further, there are only OBs at the top of the array for purposes of simplicity.

In accordance with this aspect, pointer P4320does not need to be confined to the FPN OB rows304. Rather than have pointer P4320cycle the FPN OB window, pointer P4320could extend its region of activity to rows which are never read (e.g., rows not visited by pointer P1, pointer P2, or pointer P3).

In some aspects, rows that neighbor the read rows are not reset thereby allowing signal to continue to accumulate, which can create an issue as the rows may bloom into the first and last read rows either in the vertical clear or vertical OB windows. In accordance with an aspect, pointer P4320cycles all of the unread rows and jumps back to the start of the FPN OB window when it needs to, to stay one line ahead (in time) of pointer P1316. It should be noted that if there are more unread rows in the array than read rows, not all rows may be reset unless, having finished the FPN OB rows, pointer P4320returns to the position from which it jumped and continues its cycle. In this case, each unread row will be reset less than once per frame on average (as opposed to never being reset).

In the case of skipping rows (and/or columns) to facilitate image preview capability without having to read through the entire sensor, pointer P4320could also be used to reset the intermediate rows in between the sampled rows. Further, physical rows which are served by pointer P4320are not visited by pointer P2318or pointer P3. Therefore, the risk of pointer collision is mitigated.

FIG. 7illustrates a fifth exemplary implementation700that incorporates anti-blooming features and synchronous operation, according to an aspect. For purposes of explanation, when the pointers are not visible, the pointers are assumed to be active in a small subset of dummy rows. The pixels indicated by rows702are reset by pointer P4320to protect the active windows from blooming.

In this example, there is no jumping of pointer P4320. Pointer P4320is only used to reset a small number of extra rows around the active windows to protect against blooming. Further, pointer P4320can be synchronized to the frame, which can provide for a simpler RTL (resistor-transistor logic) coding embodiment while maintaining various advantages.

When pointer P4320is idle, pointer P4320continuously cycles around a small subset of dummy rows (similar to pointer P2318) in order to maintain balanced activity profile throughout the frame.

In view of the aspects shown and described above, methodologies that may be implemented in accordance with the disclosed subject matter, will be better appreciated with reference to the following flow charts. While, for purposes of simplicity of explanation, the methodologies are shown and described as a series of blocks, it is to be understood and appreciated that the disclosed aspects are not limited by the number or order of blocks, as some blocks may occur in different orders and/or at substantially the same time with other blocks from what is depicted and described herein. Moreover, not all illustrated blocks may be required to implement the methodologies described hereinafter. It is to be appreciated that the functionality associated with the blocks may be implemented by software, hardware, a combination thereof or any other suitable means (e.g. device, system, process, component). Additionally, it should be further appreciated that the methodologies disclosed hereinafter and throughout this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methodologies to various devices. Those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram.

FIG. 8illustrates a method800for improved optically black reference pixels in a complementary metal-oxide-semiconductor (CMOS) imaging system-on-chip (iSoC) sensor, according to an aspect. In accordance with some aspects, method can improve quality of optically black reference pixels by enabling shortening of the integration time relative to the active pixel integration time. At802, a pointer P4is placed in idle mode when column fixed pattern noise (FPN) optical black (OB) rows are not being read. At804, pointer P4is implemented one line before pointer P1arrives at a first column FPN OB row. At806, pointer P2and pointer P3are diverted away from the column FPN OB rows when pointer P2and pointer P3reach the column FPN OB rows.

In accordance with some aspects, placing pointer P4in idle mode includes allocating a first subset of dummy rows to pointer P4and a second subset of dummy rows to pointer P2. The first subset of dummy rows are independent of the second subset of dummy rows. Placing pointer P4in idle mode also includes sending pointer P4to first subset of dummy rows when the column FPN OB rows are not being read.

According to some aspects, method800can include continuously rotating pointer P4within the column FPN OB rows. In accordance with another aspect, method800can include jumping pointer P4on the second cycle, wherein pointer P4arrives at the first FPN OB row one line before pointer P1arrives at the first FPN OB row.

In another aspect, method can include extending an activity region of pointer P4to rows that are not read. Further to this aspect, method800can include cycling pointer P4within the rows that are not read and jumping pointer P4to a start of the FPN OB window one line ahead of pointer P1.

In accordance with some aspects, method800can include using pointer P4to reset a set of extra rows around active windows. Further to this aspect, method800includes synchronizing pointer P4to a frame.

According to some aspects, the integration time for the OB pixels can be made smaller than the integration time for actual imaging pixels. The shorter integration time can reduce dark current accumulation and concomitant noise, thereby improving the optical black pixels by increasing the accuracy for determining FPN offset values based on the OB processing.

Referring now toFIG. 9, illustrated is a block diagram of an exemplary digital camera system operable to execute the disclosed architecture, according to an aspect. In order to provide additional context for various aspects of the various embodiments,FIG. 9and the following discussion are intended to provide a brief, general description of a suitable electronic computing environment900in which the various aspects of the various embodiments can be implemented. Additionally, while the various embodiments described above may be suitable for application in the general context of instructions that may run or be executed in conjunction with an electronic device, those skilled in the art will recognize that the various embodiments also can be implemented in combination with other program modules and/or as a combination of hardware and software.

Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks associated with electronic computing environment900. Moreover, those skilled in the art will appreciate that the disclosed aspects can be practiced with other electronic system configurations, including hand-held computing devices, microprocessor-based or programmable consumer electronics, single-processor or multiprocessor state machines, minicomputers, as well as personal computers, and the like, each of which can be operatively coupled to one or more associated devices.

The illustrated aspects of the various embodiments may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a wired or wireless communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

An electronic processing device typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the electronic processing device and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media can include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, or any other medium which can be used to store the desired information and which can be accessed by the electronic processing device.

Continuing to referenceFIG. 9, the exemplary electronic processing environment900for implementing various aspects of one or more of the various embodiments includes a digital camera902, the digital camera902includes a system processor904, optics906, an image sensor908, an output circuit910, a signal processing circuit912, a system memory914and a system bus916. The system bus916couples to system components including, but not limited to, the system memory914to the system processor904. The system processor904can be a suitable semiconductor processing device manufactured for digital camera902, or any of various commercially available processors. Dual microprocessors and other multi-processor architectures may also be employed as the system processor904.

Optics906can comprise one or more lens elements comprised of refractive material. The refractive material can be suitable to refract electromagnetic radiation, particularly in the visible spectrum, but also the near infrared or ultraviolet spectra, or other suitable spectra. Particularly, optics906can be configured to project and focus an image of an object onto image sensor908. Optics can also be configured with an actuator (not depicted) to mechanically adjust optics906to focus objects at varying distances from digital camera902.

Image sensor908can comprise any of various sensors for receiving electromagnetic radiation and generating electric signals proportionate to a magnitude of the electromagnetic radiation. For instance, image sensor908can comprise a video tube, a charge-coupled device, or a CMOS device, or the like, or an array of such devices. In a particular example, image sensor908can comprise an array of photodetectors. Electric signals generated by image sensor908can be transferred to output circuit910, in response to a clock signal generated by an electronic clock(s)918managed by system processor904. The electric signals can then be output to signal processing circuit912for image processing.

Signal processing circuit912can be any suitable hardware or software processing entity, including an integrated circuit(s), an application specific integrated circuit(s) (ASIC), a state machine, or other suitable signal processing device. Signal processing circuit912can be configured to perform operations on electric signals provided by output circuit910. These operations can include correlated double sampling, gamma processing, analog to digital conversion, gain adjustment, interpolation, compression, or a combination thereof or of the like, to generate digital data to be adapted for presentation on an electronic display920of digital camera902. Additionally, signal processing circuit912can store the digital data in system memory914before, during and after the operations.

The system bus916can be any of several types of bus structure suitable for communicatively connecting components of digital camera902. System bus916can further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory914can include read-only memory (ROM)922and random access memory (RAM)924. A basic input/output system (BIOS) for digital camera902can be stored in a non-volatile memory such as ROM, EPROM, EEPROM, which BIOS contains the basic routines that help to transfer information between elements within the digital camera902, when powered on for instance. The RAM924can include a high-speed RAM such as static RAM for caching data. Furthermore, digital camera can include removable memory926, which can include any suitable non-volatile memory (e.g., Flash memory), or other removable memory technology.

A number of program modules can be stored in the system memory914, including an operating system928, one or more application programs or program modules930and program data932. All or portions of the operating system, applications, modules, and/or data can also be cached in the RAM924. It is appreciated that the various embodiments can be implemented with various commercially available or proprietary operating systems or combinations of operating systems.

The display920is connected to the system bus916via an interface, such as a video adapter934. Display920can comprise a flat panel display, such as a liquid crystal display, a light-emitting diode display, or the like. System processor904can control output of image data to present a digital replica of the image received by image sensor908on display920. In addition, digital camera902can output the image data to an external display936via a suitable external interface938.

A user can enter commands and information (e.g., user input940) and/or other external input942can be entered into the digital camera902through one or more input devices, e.g., touch screen buttons, switches, dials, levers, etc. For instance, zoom functionality is often implemented by pressing a button, dial, lever, etc., in one direction to zoom in, or another direction to zoom out. Further, display options, selection of images, and similar display commands can be input via a touch screen, often implemented as part of display920. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, or the like. These and other input devices are often connected to the system processor904through an input device interface944that is coupled to the system bus916, but can be connected by other interfaces, such as a parallel port, an IEEE1394 serial port, a game port, a USB port, an IR interface, a Bluetooth interface, etc.

The external interface938can include at least one or both of Universal Serial Bus (USB) and IEEE1394 interface technologies. Other external connection technologies are within contemplation of the subject matter claimed herein. Moreover, external interface938can include a wireless technology, such as a Wi-Fi communication technology, Bluetooth™ technology, infrared (IR) technology, cellular technology, or the like. In addition to an external display, external interface938can facilitate communicatively coupling digital camera902to one or more remote devices946. Remote device(s)946can include a computer, a display, a memory or storage device948, and so on. Moreover, commands can be given to digital camera902from remote device(s)942over external interface938to system processor904. This can facilitate remote control of digital camera902, for remote camera operation (e.g., taking pictures, adding or deleting pictures from system memory914, etc.), transferring data, such as stored digital images, updating operation system928, applications/program modules930, or data932, and so on.

Wi-Fi, or Wireless Fidelity, allows connection to the Internet from various locations within range of a WiFi access point, without wires. Wi-Fi is a wireless technology similar to that used in a cell phone that enables such devices, e.g., computers, to send and receive data indoors and out; within the range of the access point. Wi-Fi networks use radio technologies called IEEE 802.11 (a, b, g, n, etc.) to provide secure, reliable, fast wireless connectivity. A Wi-Fi network can be used to connect computers to each other, to the Internet, and to wired networks (which use IEEE 802.3 or Ethernet). Wi-Fi networks operate in the unlicensed 2.4 and 5 GHz radio bands, at an 11 Mbps (802.11a) or 54 Mbps (802.11b) data rate, for example, or with products that contain both bands (dual band), so the networks can provide real-world performance similar to the basic 10BaseT wired Ethernet networks used in many offices.

FIG. 10is a schematic block diagram of a sample-computing environment1000with which the disclosed embodiments can interact, according to an aspect. The system1000includes one or more client(s)1010. The client(s)1010can be hardware and/or software (e.g., threads, processes, computing devices). The system1000also includes one or more server(s)1020. The server(s)1020can be hardware and/or software (e.g., threads, processes, computing devices). The servers1020can house threads to perform transformations by employing the subject innovation, for example.

One possible communication between a client1010and a server1020can be in the form of a data packet adapted to be transmitted between two or more computer processes. The system1000includes a communication framework1040that can be employed to facilitate communications between the client(s)1010and the server(s)1020. The client(s)1010are operably connected to one or more client data store(s)1050that can be employed to store information local to the client(s)1010. Similarly, the server(s)1020are operably connected to one or more server data store(s)1030that can be employed to store information local to the servers1020.

In the subject specification and annexed drawings, terms such as “store,” “data store,” “data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component, refer to “memory components,” or entities embodied in a “memory” or components comprising the memory. It will be appreciated that the memory components described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.

Various aspects or features described herein can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. In addition, various aspects disclosed in the subject specification can also be implemented through program modules stored in a memory and executed by a processor, or other combination of hardware and software, or hardware and firmware.

What has been described above includes examples of systems and methods that provide advantages of the one or more aspects. It is, of course, not possible to describe every conceivable combination of components or methods for purposes of describing the aspects, but one of ordinary skill in the art may recognize that many further combinations and permutations of the claimed subject matter are possible. Furthermore, to the extent that the terms “includes,” “has,” “possesses,” and the like are used in the detailed description, claims, appendices and drawings such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.