TIME DELAY INTEGRATION SENSOR WITH PIXELS HAVING DIFFERENT SENSING ABILITY

The present disclosure provides a time delay integration (TDI) sensor using a rolling shutter. The TDI sensor includes multiple pixel columns. Each pixel column includes multiple pixels arranged in an along-track direction, wherein two adjacent pixels or two adjacent pixel groups in every pixel column have a separation space therebetween. The separation space is equal to a pixel height multiplied by a time ratio of a line time difference of the rolling shutter and a frame period, or equal to a summation of at least one pixel height and a multiplication of the pixel height by the time ratio of the line time difference and the frame period. The TDI sensor generates image frames using pixels having different sensing ability for a processor to perform image combination.

To the extent any amendments, characterizations, or other assertions previously made (in this or in any related patent applications or patents, including any parent, sibling, or child) with respect to any art, prior or otherwise, could be construed as a disclaimer of any subject matter supported by the present disclosure of this application, Applicant hereby rescinds and retracts such disclaimer. Applicant also respectfully submits that any prior art previously considered in any related patent applications or patents, including any parent, sibling, or child, may need to be re-visited.

BACKGROUND

1. Field of the Disclosure

This disclosure generally relates to a time delay integration (TDI) sensor and, more particularly, to a TDI Complementary Metal-Oxide-Semiconductor (CMOS) image sensor that implements the rolling shutter operation by spatial compensation.

2. Description of the Related Art

The time delay integration (TDI) sensor uses an area array image sensor to capture images from an imaging platform that is moving relative to the imaged object or scene at a constant speed. The TDI sensor is conceptually considered as the stack of linear arrays, wherein each linear array moves across a same point of the scene at a time period that the image sensor moves a distance of one pixel.

Conventionally, the charge-coupled device (CCD) technology has been used for TDI applications because CCDs intrinsically operate by shifting charge from pixel to pixel across the image sensor to allow charges between pixels to integrate when the image sensor moves across a same point of the imaged scene. However, CCD technology is relatively expensive to fabricate and CCD imaging devices consume relatively high power.

Although using a CMOS circuit can achieve lower power, higher degree of integration and higher speed, the existing designs suffer from higher noises. Although a 4-transistor (4T) structure can be used to minimize noises, the 4T pixels are clocked using a rolling shutter technique. Using the rolling shutter clocking can cause artifacts in the captured image since not all pixels are integrated over the same time period.

Therefore, U.S. Pat. No. 9,148,601 provides a CMOS image sensor for TDI imaging. Please refer toFIG.1, the CMOS image sensor includes multiple pixel columns112, and each pixel column is arranged to be parallel to an along-track direction Da_t. For compensating the integration interval of the rolling shutter of the CMOS image sensor, a physical offset150is further arranged between two adjacent pixels of each pixel column112, wherein if the pixel column112has N rows, each physical offset150is equal to a pixel height divided by N.

Accordingly, the present disclosure further provides a TDI CMOS image sensor that implements the rolling shutter operation by spatial compensation.

SUMMARY

The present disclosure provides a TDI CMOS image sensor with a separation space determined according to the pixel height, the line time difference of a rolling shutter and the frame period.

The present disclosure further provides a TDI CMOS image sensor that changes the line time difference corresponding to different conditions with a fixed separation space.

The present disclosure further provides a TDI CMOS image sensor that arranges two separately operated pixel arrays in an along-track direction to increase a number of times of integrating pixel data corresponding to the same position of a scene.

The present disclosure provides a TDI CMOS image sensor that captures an image frame using a rolling shutter and moves with respect to a scene in an along-track direction. The image sensor includes a pixel array, multiple first integrators and multiple second integrators. The pixel array has multiple pixel columns, each of the pixel columns comprising multiple pixels arranged in the along-track direction, and two adjacent pixels of each of the pixel columns having a separation space therebetween, wherein the multiple pixels of each of the pixel columns comprise multiple first pixels and multiple second pixels, and the multiple first pixels are exposed by a first exposure time and the multiple second pixels are exposed by a second exposure time, different from the first exposure time. The multiple first integrators respectively integrate pixel data of the multiple first pixels. The multiple second integrators respectively integrate pixel data of the multiple second pixels.

The present disclosure provides a TDI CMOS image sensor that captures an image frame using a rolling shutter and moves with respect to a scene in an along-track direction. The image sensor includes a pixel array, multiple first integrators and multiple second integrators. The pixel array has multiple pixel columns, each of the pixel columns comprising multiple pixels arranged in the along-track direction, and two adjacent pixels of each of the pixel columns having a separation space therebetween, wherein the multiple pixels of each of the pixel columns comprise multiple first pixels and multiple second pixels, and the multiple first pixels have a first quantum efficiency and the multiple second pixels have a second quantum efficiency, different from the first quantum efficiency. The multiple first integrators respectively integrate pixel data of the multiple first pixels. The multiple second integrators respectively integrate pixel data of the multiple second pixels.

The present disclosure provides a TDI CMOS image sensor that captures an image frame using a rolling shutter and moves with respect to a scene in an along-track direction. The image sensor includes a pixel array, multiple first integrators and multiple second integrators. The pixel array has multiple pixel columns, each of the pixel columns comprising multiple pixels arranged in the along-track direction, and two adjacent pixels of each of the pixel columns having a separation space therebetween, wherein the multiple pixels of each of the pixel columns comprise multiple first pixels and multiple second pixels, the multiple first pixels are exposed by a first exposure time and have a first quantum efficiency, and the multiple second pixels are exposed by a second exposure time, different from the first exposure time and have a second quantum efficiency, different from the first quantum efficiency. The multiple first integrators respectively integrate pixel data of the multiple first pixels. The multiple second integrators respectively integrate pixel data of the multiple second pixels.

In the present disclosure, the separation space is not directly related to a size of the pixel array (i.e. a number of pixels), and the separation space can be determined once the frame period and the line time difference are determined.

DETAILED DESCRIPTION OF THE EMBODIMENT

The CMOS image sensor of the present disclosure compensates a line time difference in time delay integration (TDI) imaging using a rolling shutter by arranging a separation space between pixels in an along-track direction. Accordingly, pixel data corresponding to the same position of an imaged scene is integrated in successive image frames so as to increase the signal-to-noise ratio (SNR), wherein a number of integration is related to a size of pixel array.

The concept of TDI imaging is known to the art, and the present disclosure is to eliminate the imaging distortion generated in a TDI CMOS image sensor using rolling shutter technique.

Please refer toFIG.2, it is a schematic diagram of a TDI CMOS image sensor200according to a first embodiment of the present disclosure. The TDI CMOS image sensor200captures image frames using a rolling shutter, and moves toward an along-track direction Da_twith respect to a scene, wherein the scene is determined according to an application of the TDI CMOS image sensor200. For example, when the TDI CMOS image sensor200is applied to a scanner, the scene is a scanned document; whereas, when the TDI CMOS image sensor200is applied to a satellite or aircraft, the scene is a ground surface.

The operation of the rolling shutter is known to the art, and thus details thereof are not described herein.

The TDI CMOS image sensor200includes a pixel array21. The pixel array21includes multiple pixel columns212. Each of the pixel columns212includes multiple pixels2123(e.g., shown as regions filled with slant lines herein) arranged in the along-track direction Da_t(e.g., shown as a longitudinal direction of the pixel array21). Two adjacent pixels of each pixel column212have a separation space2124(e.g., shown as blank regions herein) therebetween.

Please refer toFIG.3, it is an operational schematic diagram of the TDI CMOS image sensor200ofFIG.2. In one aspect, the separation space2124is equal to a multiplication of a pixel height W of one pixel2123in the along-track direction Da_tby a time ratio of a line time difference t of the rolling shutter and a frame period T of capturing the image frame (e.g.,FIG.3showing three image frames), i.e. separation space=W×t/T.

In the present disclosure, the line time difference t is a time interval between a time of starting or ending exposure of two adjacent pixel rows.

InFIG.3, it is assumed that the scene includes 3 positions or objects A, B and C moving rightward (i.e. along-track direction Da_t). Stage1and Stage2indicate two pixel rows of each pixel column212, wherein the separation space W×t/T is arranged between Stage1and Stage2. In the present disclosure, the frame period T is determined according to brightness of the scene and a sensitivity of the pixel array21. A moving speed of the TDI CMOS image sensor200is set as the pixel height W divided by the frame period T.

BecauseFIG.3assumes that the pixel column212of the pixel array21has two pixel rows, the frame period T, in which the TDI CMOS image sensor200captures one image frame, includes two line times, which have a line time difference t. Herein, a line time is referred to a processing time interval for accomplishing the exposing and reading of one pixel row. For example,FIG.3shows that a first image frame includes two pixel rows F1_1and F1_2; a second image frame includes two pixel rows F2_1and F2_2; and a third image frame includes two pixel rows F3_1and F3_2.

In this embodiment, the TDI CMOS image sensor200further includes multiple integrators, e.g.,FIG.3showing two integrators31and32, wherein the integrators are, for example, a buffer (i.e. digital integrator) or a capacitor (i.e. analog integrator), and a number of the integrators are preferably corresponding to a number of pixel columns212so as to determine a width of the imaged scene. The integrators31and32are respectively used to integrate pixel data in adjacent image frames corresponding to a same position or object of the scene.

For example, in the first image frame (e.g., including F1_1and F1_2), Stage1senses pixel data of the position or object A of the scene, and integrates (or adds) to the integrator31, e.g., shown as IA; now, the integrator32does not yet integrate (or store) any pixel data, e.g., shown as 0.

As the scene moves in the along-track direction Da_tat a speed W/T, in the second image frame (e.g., including F2_1and F2_2), Stage1senses pixel data of the position or object B of the scene, and integrates (or adds) to the integrator32, e.g., shown as IB; and Stage2senses pixel data of the position or object A of the scene, and integrates (or adds) to the integrator31, e.g., shown as 2IA(indicating integrated by two times).

As the scene continuously moves in the along-track direction Da_tat the speed W/T, in the third image frame (e.g., including F3_1and F3_2), the pixel data 2IAassociated with the object A already integrated in the integrator31is read out at first. Next, Stage1senses pixel data of the position or object C of the scene, and integrates (or adds) to the integrator31, e.g., shown as IC; and Stage2senses pixel data of the position or object B of the scene, and integrates (or adds) to the integrator32, e.g., shown as 2IB(indicating integrated by two times). When the scene is continuously imaged, the TDI CMOS image sensor200continuously integrates and reads pixel data using the process as shown inFIG.3to improve the SNR of the captured image frame.

In one aspect, the frame period T (i.e. exposure interval of one image frame) is larger than a summation of row exposure times for capturing all pixel rows of the pixel array21using the rolling shutter, e.g.,FIG.3showing that an extra time textrais left after a second pixel row of every image frame is exposed and read.

In one non-liming aspect, within a time difference (i.e. textra) between the frame period T and the summation of row exposure times, the image sensor200enters a sleep mode to save power.

In one non-liming aspect, a column analog-to-digital converter (ADC) (e.g., included in the readout circuit23) of the TDI CMOS image sensor200performs, within the time difference textra, the analog-digital (AD) conversion on pixel signals of auxiliary pixels (e.g., dark pixels), external voltages or temperatures of an external temperature sensor of the pixel array21. More specifically, within the time difference textra, the column ADC is used to perform the AD conversion on sensing signals outside the pixel columns212so as to broaden applications of the TDI CMOS image sensor200. In this aspect, a line time is preferably set as the minimum time required for processing one row of pixel data.

In this embodiment, the readout circuit23samples every pixel using, e.g., correlation double sampling (CDS).

Please refer toFIG.2again, in another aspect, the separation space2124is equal to a summation of a pixel height W in the along-track direction Da_tand a multiplication of the pixel height W by a time ratio of a line time difference t of the rolling shutter and a frame period T of capturing the image frame, i.e. separation space=W×(y+t/T).

Please refer toFIG.4Atogether, it is another operational schematic diagram of the TDI CMOS image sensor200ofFIG.2. InFIG.4A, it is assumed that one scene includes eight positions or objects A to H, and moves rightward (i.e. along-track direction Da_t). Stage1to Stage4indicate four pixel rows of one pixel column212, wherein the separation space W×(y+t/T) is arranged between two adjacent pixels, wherein y=0 or a positive integer.FIG.4Ashows an aspect that y=1; and an aspect of y=0 is shown inFIG.3.

BecauseFIG.4Aassumes that the pixel array21includes four pixel rows, thus the frame period T of the TDI CMOS image sensor200for capturing one image frame includes four line times, which have a line time difference t from each other. For example,FIG.4Ashows that one image frame includes four pixel rows F1_1to F1_4; a next image frame includes four pixel rows F2_1to F2_4; and a further next image frame includes four pixel rows F3_1to F3_4; and so on.

Similarly, the TDI CMOS image sensor200further includes multiple integrators, e.g.,FIG.4Ashowing four integrators41to44. The integrator41is used to integrate pixel data in a first image frame (e.g., frame including F1_1to F1_4) and a second image frame (e.g., frame including F3_1to F3_4) corresponding to the same position (e.g., position or object F) of the scene, wherein the first image frame and the second image frame is separated by one image frame (e.g., frame including F2_1to F2_4). The operations of other integrators42to44are identical to that of the integrator41, and the difference is in integrating the pixel data at different positions or objects.

It is seen fromFIG.4Athat a first pixel (e.g., Stage1) in the first image frame for sensing pixel data (e.g., IF) of the same position (e.g., F) and a second pixel (e.g., Stage2) in the second image frame for sensing pixel data (e.g., IF) of the same position (e.g., F) are two adjacent pixels of the same pixel column212in the pixel array21. Therefore, the integrators (e.g.,41to44) do not integrate the pixel data IFin the first pixel and the second pixel corresponding to the same position within a frame period of the one image frame between the first image frame and the second image frame. The sensing and integration of positions or objects D and B are shown by dashed lines and arrows inFIG.4A.

In the aspect ofFIG.4A, because the integrators41to44integrate pixel data in the image frames separated by one image frame (e.g., frame including F2_1to F2_4) corresponding to the same position or the same object of a scene, if it is assumed that the pixel columns212have N pixels, the integrators41to44integrate N/2 times of pixel data corresponding to the same position or the same object of the scene.

The pixel data of the image frame F2_1to F2_4is integrated in another group of integrators, wherein the pixel data of the same position or the same object of the scene is also integrated by skipping one image frame (e.g., frame including F3_1to F3_4).

When y=n, a same position of the scene is sensed by a next adjacent pixel of the same pixel column212after n image frames. Once the control signal outputted by the control circuit27is properly arranged, the pixel data of the same position or object of the scene is accurately integrated in the same integrator.

In addition, in the aspect ofFIG.4A, because adjacent pixels of the pixel columns212have a larger separation space2124, in the case that a wider imaged scene image is required, it is possible to arrange buffers in the separation space2124every predetermined number of pixel columns to buffer or amplify control signals of the pixel row. For example as shown inFIG.4B, in the separation space2124, the buffers49are arranged to buffer or amplify pixel control signals, e.g., including the reset signal Srst, signal transfer signal Sgt and row selection signal Srs, but not limited to. In this way, even a pixel array having a large number of pixel columns can still operate accurately.

Please refer toFIG.5, it is a schematic diagram of a TDI CMOS image sensor500according to a second embodiment of the present disclosure. The TDI CMOS image sensor500is also captures an image frame using a rolling shutter, and moves toward an along-track direction Da_twith respect to a scene.

The TDI CMOS image sensor500includes a pixel array51. The pixel array51includes multiple pixel columns512each including multiple pixels arranged in the along-track direction Da_t. A separation space5124is arranged between two adjacent pixel groups to compensate a line time difference in using the rolling shutter, wherein each pixel group includes a first pixel5123and a second pixel5215directly connected to each other, i.e. no separation space5124therebetween.

The TDI CMOS image sensor500further includes a first readout circuit53and a second readout circuit55. As shown inFIG.5, the first readout circuit53is coupled to multiple first pixels5123in the pixel columns512via a readout line513so as to read pixel data of the first pixels5123, and the second readout circuit55is coupled to multiple second pixels5125in the pixel columns512via a readout line515so as to read pixel data of the second pixels5125.

Please refer toFIG.6, it shows an operational schematic diagram of the TDI CMOS image sensor500inFIG.5. In one aspect, the separation space5124is a multiplication of a pixel height W in the along-track direction Da_tby a time ratio of a line time difference t of the rolling shutter and a frame period T of capturing the image frame (e.g.,FIG.6showing two image frames), i.e. separation space=W×t/T.

InFIG.6, it is assumed that a scene includes eight positions or objects A to H, and moves rightward (i.e. along-track direction Da_t).

Because it is assumed that the pixel array51inFIG.6has four pixel groups in the along-track direction Da_t, a frame period T that the TDI CMOS image sensor500captures one image frame includes 4 line times, which have a line time difference t between each other. For example,FIG.6shows that a first image frame includes four rows of pixel groups F1_1to F1_4; and a second image frame includes four rows of pixel groups F2_1to F2_4.

In this embodiment, the first pixel5123and the second pixel5125of each pixel group are exposed simultaneously, and the pixel data thereof is respectively integrated by the first readout circuit53and the second readout circuit55simultaneously.

For example, in the line time of F1_2of a first image frame (e.g., frame including F1_1to F1_4), Stage3and Stage4are exposed at the same time, and pixel data of Stage3(e.g., ID) is integrated by the first readout circuit53to the integrator63, and pixel data of Stage4(e.g., IC) is integrated by the second readout circuit55to the integrator64. In the line time of F1_3of the first image frame, Stage5and Stage6are exposed at the same time, and pixel data of Stage5(e.g., IB) is integrated by the first readout circuit53to the integrator65, and pixel data of Stage6(e.g., IA) is integrated by the second readout circuit55to the integrator66. The exposure and integration of other line times in a frame period T of the first image frame are similar to the line times F1_2and F1_3.

For example, in the line time of F2_3of a second image frame (e.g., frame including F2_1to F2_4), Stage5and Stage6are exposed at the same time, and pixel data of Stage5(e.g., IC) is integrated by the first readout circuit53to the integrator64, shown as 2ICindicating integrated by two times; and pixel data of Stage6(e.g., IB) is integrated by the second readout circuit55to the integrator65, shown as 2IBindicating integrated by two times. The exposure and integration of other line times in a frame period T of the second image frame are similar to the line times F2_3.

For example, the first readout circuit53and the second readout circuit55are respectively coupled to each integrator via a switching device (e.g., a multiplexer, but not limited thereto). The switching device is controlled by a control signal (e.g., generated by the control circuit57) to integrate pixel data read by the first readout circuit53or the second readout circuit55to the same integrator. It is appreciated thatFIG.6shows only a part of integrators for describing the present disclosure.

More specifically, multiple integrators of the TDI CMOS image sensor500respectively store pixel data in the first image frame (e.g., frame including F1_1to F1_4) and the second image frame (e.g., frame including F2_1to F2_4), adjacent to each other, corresponding to the same position (e.g., B) of a scene, wherein in the first image frame, pixel data (e.g. IB) corresponding to a same position (e.g., B) of the scene is read by the first readout circuit53and integrated to an integrator65; and in the second image frame, the pixel data (e.g. IB) corresponding to the same position (e.g., B) of the scene is read by the second readout circuit55and integrated to the integrator65. As long as the output signal of the control circuit57is corresponding arranged, the pixel data read from different readout circuits is correctly integrated in the same integrator. The method of integrating pixel data of associated pixels by other integrators is similar to the descriptions in this paragraph, and thus is not repeated herein.

In other aspects, the above embodiments ofFIG.2andFIG.5are combinable. For example, a separation space between two adjacent pixel groups is a summation of a pixel height W and a multiplication of the pixel height W by a time ratio of a line time difference t of the rolling shutter and a frame period T of capturing the image frame, i.e. separation space=W×(y+t/T).

As mentioned above, the TDI CMOS image sensor integrates pixel data for multiple times using integrators to increase the SNR. However, in order to allow both bright regions and dark regions in one image frame to be within a suitable gray level range so as to increase a dynamic range, one combination image is obtained by combining two image frames acquired by pixels having different sensing ability. The present disclosure further provides a TDI CMOS image sensor that integrates pixel data of two image frames acquired using two different quantum efficiency and/or exposure times.

As shown inFIG.11, pixels having high sensing ability are used to acquire light having lower intensity in order not to cause overexposure, and pixels having low sensing ability are used to acquire light having higher intensity in order not to cause underexposure. After synthesizing output signal, it is able to obtain a higher dynamic range than a dynamic range of each of the pixels having high sensing ability and having low sensing ability.

Please refer toFIG.7, it is a schematic diagram of a TDI CMOS image sensor700according to a third embodiment of the present disclosure. The TDI CMOS image sensor700also captures an image frame using a rolling shutter and moves with respect to a scene in an along-track direction Da_t.

The TDI CMOS image sensor700includes a pixel array71, a readout circuit73, a control circuit77, multiple first integrators BL1-BLN, multiple second integrators BH1-BHNand a processor79. The processor79is a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

The pixel array71also includes multiple pixel columns112. Each of the pixel columns112includes multiple pixels (e.g., stage1to stage8) arranged in the along-track direction Da_t, and two adjacent pixels of each of the pixel columns112have a separation space150therebetween.

The control circuit77outputs control signals, e.g., including the row selection signal, reset signal and charge transfer signal, to cause the pixel array71to operate in rolling shutter, which is known to the art and thus details thereof are not described herein.

The readout circuit73reads gray level data of each pixel to be integrated in corresponding integrators BL1˜-BLNor BH1˜BHN, e.g., the multiple first integrators BL1˜BLNand the multiple second integrators BH1˜BHNcorresponding to a same pixel column respectively integrating pixel data of a same position or object of a scene in adjacent image frames.

The difference between the embodiment inFIG.7and the above embodiments is that the multiple pixels of each of the pixel columns112of the pixel array71include identical numbers of multiple first pixels (e.g., stage1to stage4) and multiple second pixels (e.g., stage5to stage8). In this embodiment, the first pixels stage1to stage4have first sensing ability and the second pixels stage5to stage8have second sensing ability, different from the first sensing ability.

The sensing ability of a pixel circuit determines intensity of output pixel data. Upon receiving identical light, if the sensing ability is larger, the intensity of output pixel data is larger; and if the sensing ability is smaller, the intensity of output pixel data is smaller.

In one aspect, the sensing ability is an exposure time. For example, the first pixels stage1to stage4are exposed by a first exposure time, and the second pixels stage5to stage8are exposed by a second exposure time, different from the first exposure time. In the case that the second exposure time is longer than the first exposure time, the second exposure time is arranged as the maximum exposure time that can be set, and the first exposure time is arranged to be smaller than, for example, a half of the second exposure time, but not limited to.

In another aspect, the sensing ability is quantum efficiency. For example, the first pixels stage1to stage4are arranged to have first quantum efficiency, and the second pixels stage5to stage8are arranged to have second quantum efficiency, different from the first quantum efficiency. Please refer toFIGS.12A to12C, they are schematic diagrams showing that the second quantum efficiency is larger than the first quantum efficiency. For example, each first pixels12_1and each second pixels12_2includes a photodiode124covered by a filter (e.g., color filter)122and at least one convex microlens121, and a light shield layer123is used to define a fill factor of each of the first pixels121and second pixels12_2. The material of the light shield layer123is not particularly limited as long as it can block light from impinging onto the photodiode124.

As shown inFIG.12A, each of the second pixels12_2is aligned with a center of the convex microlens121, and each of the first pixels12_1is aligned with edges of two adjacent convex microlens121such that the second pixels12_2(i.e. stage5to stage8) have a higher quantum efficiency than the first pixels12_1(i.e. stage1to stage4).

In another aspect, each of the second pixels12_2is aligned with a center of the convex microlens121, and each of the first pixels12_1is aligned with a center of a concave microlens such that the second pixels12_2have a higher quantum efficiency than the first pixels12_1.

As shown inFIG.12B, each of the second pixels12_2is covered by the filter122having a second transparency, and each of the first pixels12_2is covered by the filter122having a first transparency, which is lower than the second transparency such that the second pixels12_2have a higher quantum efficiency than the first pixels12_1.

As shown inFIG.12C, each of the second pixels12_2has a second fill factor, and each of the first pixels12_2has a first fill factor, which is smaller than the second fill factor such that the second pixels12_2have a higher quantum efficiency than the first pixels12_1.

To define the quantum efficiency of the first pixels12_1and the second pixels12_2, at least one arrangement ofFIGS.12A to12Cis selected.

As shown inFIG.7, in one aspect, the first pixels stage1to stage4(shown by rectangles filled with slant lines) are all adjacent to one another, and the second pixels stage5to stage8(shown by blank rectangles) are all adjacent to one another. It should be mentioned that althoughFIG.7shows that the first pixels stage1to stage4have low sensing ability and the second pixels stage5to stage8have high sensing ability, it is only intended to illustrate but not to limit the present disclosure. In another aspect, the first pixels stage1to stage4have high sensing ability, and the second pixels stage5to stage8have low sensing ability.

In the third embodiment, the multiple first integrators BL1-BLNare respectively coupled to the readout circuit73, and each of the first integrators BL1-BLNrespectively integrates pixel data of the first pixels stage1to stage4; and the multiple second integrators BH1-BHNare respectively coupled to the readout circuit73, and each of the second integrators BH1-BHNrespectively integrates pixel data of the second pixels stage5to stage8, or vice versa.

For example inFIG.7, the first integrator BL1corresponding to stage1to stage4of a first pixel column112integrates pixel data corresponding to a first position of a scene, and the second integrator BH1corresponding to stage5to stage8of the first pixel column112integrates pixel data corresponding to the first position of the scene. InFIG.7, the first integrator BL2corresponding to stage1to stage4of a second pixel column112integrates pixel data corresponding to a second position of the scene, and the second integrator BH2corresponding to stage5to stage8of the second pixel column112integrates pixel data corresponding to the second position of the scene, and so on.

In the third embodiment, each of the first integrators BL1-BLNand the second integrators BH1-BHNintegrates pixel data of a same position for four times. However, a number of times of integrating pixel data by the integrators BL1-BLNand BH1-BHNis determined according to a number of first pixels (or activated first pixels, i.e. not every first pixels being activated in operation) and a number of second pixels (or activated second pixels, i.e. not every second pixels being activated in operation) in the same column.

The processor79receives pixel data in the first integrators BL1-BLNintegrated within one frame period to form a first image frame, and receives pixel data in the second integrators BH1-BHNintegrated within another frame period to form a second image frame. The processor79then combines the first image frame and the second image frame to form a combination image. The method of generating a combination image using two image frames may be referred to U.S. patent application Ser. No. 14/731,713 assigned to the same assignee of the present application, and the full disclosure of which is incorporated herein by reference, but the present disclosure is not limited thereto.

In another aspect, the first pixels stage1to stage4and the second pixels stage5to stage8have identical sensing ability, i.e. having identical exposure time and identical quantum efficiency. The processor79receives pixel data integrated in the first integrators BL1-BLNassociated with the first pixels stage1to stage4, and receives pixel data integrated in the second integrators BH1-BHNassociated with the second pixels stage5to stage8, and then amplifies the pixel data integrated in the first integrators BL1-BLNwith a first digital gain, and amplifies the pixel data integrated in the second integrators BH1-BHNwith a second digital gain different from (e.g., shown inFIG.7being larger than) the first digital gain. In this way, two image frames having different intensity are also generated.

In another aspect, the TDI CMOS image sensor700uses pixels having different sensing ability and different digital gains to generate a combination image.

Please refer toFIG.8, it is an alternative TDI CMOS image sensor800according to the third embodiment of the present disclosure. The difference between the TDI CMOS image sensors800and700is that the first pixels (e.g., stage1, stage3, stage5and stage7) and the second pixels (e.g., stage2, stage4, stage6and stage8) of the pixel array81are interlaced. Other parts of the TDI CMOS image sensor800are identical to the TDI CMOS image sensor700, i.e. integrating pixel data of first pixels respectively using first integrators BL1-BLN, and integrating pixel data of second pixels respectively using second integrators BH1-BHN, or vice versa. And the processor79uses a first image frame generated by the first integrators BL1-BLNand a second image frame generated by the second integrators BH1-BHNto generate a combination image.

Please refer toFIG.9, it is an alternative TDI CMOS image sensor900according to the third embodiment of the present disclosure. The difference between the TDI CMOS image sensors900and700is that the first pixels (e.g., stage1, stage2, stage5and stage6) and the second pixels (e.g., stage3, stage4, stage7and stage8) of the pixel array91are partially adjacent to each other in the along-track direction Da_t. Other parts of the TDI CMOS image sensor900are identical to the TDI CMOS image sensor700, i.e. integrating pixel data of first pixels respectively using first integrators BL1-BLN, and integrating pixel data of second pixels respectively using second integrators BH1-BHN, or vice versa. And the processor79uses a first image frame generated by the first integrators BL1-BLNand a second image frame generated by the second integrators BH1-BHNto generate a combination image.

It should be mentioned that the arrangement of first pixels and second pixels in the pixel array is not limited to those shown inFIGS.7to9as long as each pixel column having identical numbers of the first pixels and the second pixels.

Please refer toFIG.10, it is a schematic diagram of a TDI CMOS image sensor1000according to a fourth embodiment of the present disclosure. The TDI CMOS image sensor1000also captures an image frame using a rolling shutter and moves with respect to a scene in an along-track direction Da_t.

The TDI CMOS image sensor1000also includes a pixel array101, a readout circuit73, a control circuit77, multiple first integrators BL1-BLN, multiple second integrators BH1-BHNand a processor79, wherein elements identical to those ofFIGS.7to9are indicated by identical reference numerals.

Operations of the readout circuit73and the control circuit77are similar to those of the above embodiments, and thus details thereof are not repeated herein.

The pixel array101also includes multiple pixel columns112. Each of the pixel columns112includes multiple pixels (e.g., stage1to stage8) arranged in the along-track direction Da_t, and two adjacent pixels of each of the pixel columns112have a separation space150therebetween.

In this embodiment, the multiple pixels of each pixel column112include a first number of (e.g., two) multiple first pixels (e.g., stage1to stage2) and a second number (e.g., six) of, larger than the first number, multiple second pixels (e.g., stage3to stage8).

The multiple first integrators BL1-BLNare respectively coupled to the readout circuit73, and each of the first integrators BL1-BLNrespectively integrates pixel data of the first pixels stage1to stage2of the corresponding pixel column112. The multiple second integrators BH1-BHNare respectively coupled to the readout circuit73, and each of the second integrators BH1-BHNrespectively integrates pixel data of the second pixels stage3to stage8of the corresponding pixel column112. As mentioned above, the first integrators BL1-BLNand the second integrators BH1-BHNcoupled to the same pixel column112respectively integrate pixel data of a same position or object of a scene in adjacent image frames. For example, stage1to stage8of the same pixel column112integrate pixel data of a same position or object of a scene.

The processor79then generates a combination image according to a first image frame generated by the first integrators BL1-BLNand a second image frame generated by the second integrators BH1-BHN.

The processor79receives integrated pixel data from multiple first integrators BL1-BLNand multiple second integrators BH1-BHN. Because a number of integration times of the integrated pixel data of the first integrators BL1-BLNis less than a number of integration times of the integrated pixel data of the second integrators BH1-BHN, the processor79further amplifies the first pixel data using a ratio (second number/first number)=6/2 before combining images to cause the first pixel data to have a similar integrating effect to that of the second pixel data.

Next, the processor79generates a combination image using the ratio-amplified first image frame and the second image frame. In another aspect, if a number of times of integrating the first pixel data is larger than a number of times of integrating the second pixel data (i.e. a number of first pixels larger than a number of second pixels), the processor79amplifies the second pixel data using a ratio (first number/second number) or reduces the first pixel data using the ratio (first number/second number) to have similar effect of integration.

It is appreciated that a ratio between the first pixels and the second pixels inFIG.10is not limited to three times. A ratio of numbers of multiple first pixels and multiple second pixels in each pixel column112is selected according to different applications without particular limitations as long as pixel data is amplified by a ratio of pixel numbers.

In addition, althoughFIG.10shows that the first pixels stage1to stage2and the second pixels stage3to stage8are all adjacent to one another, the present disclosure is not limited thereto. In other aspects, the first pixels stage1to stage2and the second pixels stage3to stage8are interlaced or partially adjacent to each other similar to those shown inFIG.8andFIG.9. Furthermore, the position arrangement of the first pixels and the second pixels in the pixel array101are not limited to that shown inFIG.10.

It should be mentioned that althoughFIGS.7to10are illustrated using the pixel array inFIG.1, i.e. the separation space being a multiplication of a pixel height in the along-track direction Da_twith a time ratio of a line time difference of the rolling shutter and a frame period of capturing the image frame, the present disclosure is not limited thereto. Embodiments ofFIGS.7to10are also adaptable to the pixel array200inFIG.2, i.e. the separation space being a summation of a pixel height in the along-track direction Da_tand a multiplication of the pixel height with a time ratio of a line time difference of the rolling shutter and a frame period of capturing the image frame as long as the TDI CMOS image sensors inFIGS.7to10include two groups of integrators to respectively integrate pixel data of multiple first pixels and multiple second pixels having different sensing ability.

As mentioned above, the line time difference is a time interval between time points of starting exposure of two adjacent pixel rows of a pixel array.

It is appreciated that values, e.g., including a number of pixels, integrators and image frames, in every embodiment and drawing of the present disclosure are only intended to illustrate but not to limit the present disclosure.

It should be mentioned that althoughFIGS.7-10show that pixel data to be integrated into the first integrators BL1to BLNand into the second integrators BH1to BHNare read by the same readout circuit73, it is only intended to illustrate but not to limit the present disclosure. In another aspect, the pixel data to be integrated into the first integrators BL1to BLNis read by a first readout circuit, and the pixel data to be integrated into the second integrators BH1to BHNis read by a second readout circuit, different from the first readout circuit, similar to readout circuits53and55inFIG.5.

As mentioned above, when the CMOS image sensor adopting rolling shutter technique is applied to TDI imaging, the integrated pixel data is not exactly corresponding to the same position or object in a scene to generate distortion because the exposure of all pixels of a pixel array is not started and ended at the same time. Accordingly, the present disclosure further provides a TDI CMOS image sensor using a rolling shutter (e.g.,FIGS.2and5) and an operating method thereof (e.g.,FIGS.3,4A and6) that compensate the line time difference of a rolling shutter, which causes distortion, by arranging different pixel separation spaces. By arranging the control signal of a control circuit correspondingly, pixel data of corresponding position is integrated to the associated integrator correctly. Furthermore, by arranging multiple pixel arrays along an along-track direction and aligning every pixel column of the multiple pixel arrays to be able to cross the same position or object of a scene sequentially, pixel data of the aligned pixel columns can be integrated.