TND6372/D
Rev. 2, May 2023

Automotive 3 mm HDR Image Sensor
With LFM and Distance Functionality© Semiconductor Components Industries, LLC, 2022

May, 2023 − Rev. 2

1

Publication Order Number:
TND6372/D

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON ELECTRON DEVICES

1

Automotive 3 μm HDR Image Sensor With LFM
and Distance Functionality
Sergey Velichko , Dave Jasinski, Michael Guidash, Daniel Tekleab, Manuel Innocent, Andrew Perkins ,Shaheen Amanullah, Maheedhar Suryadevara, Chris Silsby, and Jeff Beck

Abstract — We described a 3 μm super-exposure pixel
1.3-MP CMOS image sensor for automotive application thatprovided for up to 140 dB high dynamic range (HDR) coverage with multiexposure operation, as well as effective
HDR with LED flicker mitigation (LFM) operation. At junction
temperatures in excess of +100 ◦ C, total signal-to-noisetransitions were kept at or above 30 dB due to further
development of the pixel pulsed gate operation and processoptimization. Pulsed LFM operation provided for capturing
flickering lights of any brightness. Two-photodiode pixel
architecture in combination with single microlens covering
the pixel also enabled distance extraction along with a color
image using in-pixel phase detect.
Index Terms — CMOS, distance, high dynamic range(HDR), image sensor, LED flicker mitigation (LFM).

Fig. 1. SE two PDs’ DCG pixel schematic.

I. I NTRODUCTION

II. P IXEL D ESIGNATEST automotive image sensors are addressing challenging street scenarios with flickering LEDs and highdynamic range (HDR). Classical multiexposure image sensor
architecture [1] was developed and HDR reached in excess of
140 dB [2] with junction temperatures up to +125 ◦C. Image
sensor architectures addressing both HDR and LED flicker
mitigation (LFM) were developed using pulsed pixel operation[3] and split-diode pixel architectures [4]–[6]. These solutions
achieved good LFM capabilities but resulted in reduced image
quality, especially at high automotive temperatures. Earlier
concept of lateral overflow integrating capacitor (LOFIC)
architecture [7], [8] was significantly changed and modified
for automotive use [9], [10] providing high-performance HDRwith LFM even at high automotive temperatures. These newautomotive image sensors are powering the latest advancements in autonomous and assisted driving as well as better
viewing applications such as surround and rear view, e-mirror,
digital video recording, and augmented reality displaying.
Some applications also require fused image and distance datausing the same camera solutions.

We designed and tested a 3 μm pixel 1.3-MP CMOS
imager that provides cinematographic HDR image quality evenfor high temperatures with all signal-to-noise ratio (SNR)
transitions at or above 30 dB. Two photodiodes (PDs) dual
conversion gain (DCG) eight-gate pixel schematic is presented
in Fig. 1.
The sensor implemented a set of operations effectively as
follows:
1) four-exposure 140 dB HDR image captures [2];2) pulsed LFM (pLFM) [3] using shutter gates SGA and
SGB, transfer gates TXA and TXB, and an in-pixel
memory node comprised of floating diffusion (FD),
DCG gate, and capacitor C;3) super-exposure (SE) LOFIC HDR LFM [9], [10] using
TXA and TXB in conjunction with the memory node;4) distance extraction along with a color image using
in-pixel phase detect by utilizing signal levels from two
PDs PDA and PDB covered by a single microlens [11].
In the pixel, reset gate (RST) and SGA/SGB gates are used
for resetting PDs PDA and PDB, as well as the memory node
FD and C. Source follower (SF) and row select (RS) gates areused to connect the pixel to the column circuitry for readout
sequences.
TCAD potential profile simulation of the pixel’s two PDs
and gates is presented in Fig. 2 confirming complete chargetransfer and PD-to-PD extremely low crosstalk characteristics.
In the simulation one, PD is being readout, while anotherL

Manuscript received January 14, 2022; revised March 22, 2022;
accepted April 1, 2022. The review of this article was arranged by Editor
G. Meynants. (Corresponding author: Sergey Velichko.)
The authors are with the Intelligent Sensing Group, onsemi, Meridian,
ID 83642 USA (e-mail: sergey.velichko@ onsemi.com).
Color versions of one or more figures in this article are available athttps://doi.org/10.1109/TED.2022.3165517.
Digital Object Identifier 10.1109/TED.2022.3165517

0018-9383 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2

IEEE TRANSACTIONS ON ELECTRON DEVICES

Fig. 2. TCAD potential profile simulation for pixel’s two PDs and gates.
(a) Side view and (b) cross-sectional view.

Fig. 4. Four-exposure room temperature 140 dB dynamic range total
SNR with HCG T1, LCG T2, LCG T3, and LCG T4 exposure readout.

Fig. 5. SE LFM pixel timing and potential diagrams.
Fig. 3. Multiexposure HDR schema with delay buffers.

PD is still holding acquired photon-generated charge. Fig. 2(a)shows the cross section of the PD being readout and some of
the RST, SF, DCG, and RS gates in the periphery. Fig. 2(b)shows the TXA/TXB gates in the center and SGA/SGB gateson the periphery, left PD is holding charge, and right PD is
being readout.
III. M ULTIEXPOSURE HDR
Multiexposure implementation followed classical HDR
schema [1] with delay buffers (Fig. 3). T1 stack and T2 stackmemory buffers are used to hold digital values of the pixel
signal acquired during first longer T1 and second shorter T2
exposures, respectively. These buffers are located in-sensor
and used during HDR per pixel signal recombination. At theend of the third shortest exposure T3, all three pixel values
were combined into an HDR multibit value, usually 20 bit forthree exposures or 24 bit for four exposures (with additional
T3 stack memory). Thus, three exposures allowed up to
120 dB and four exposures allowed up to 140 dB-HDR inframe image captures.
Characterization of four-exposure HDR mode at room temperature showed a 140 dB dynamic range and all total SNR
transitions at or above 30 dB with first T1 exposure readoutusing high conversion gain (HCG) (Fig. 4).
Combination of HCG T1 with low readout noise and low
conversion gain (LCG) T2, T3, and T4 readout with extended

dynamic range provided for both excellent low and mid-tohigh light performance while covering seven decades of lightand exposure HDR.
IV. HDR W ITH LFMWhile classical multiexposure HDR provides for excellent
performance with static sceneries, it does suffer from motionand LED flicker artifacts. We continued our previous workon the pixel SE LOFIC operation [9], [10] with further
improvement for high ambient temperatures. New timing andpotential diagrams of the pixel LOFIC HDR operation are
shown in Fig. 5. Mid-level pulsed operation of the TX andDCG gates during charge acquisition and further processoptimization reduced the memory node dark noise contribution
in comparison to earlier work [9], [10] and thus improved
SNR transitions for high temperatures. By keeping TX andDCG gate interface surfaces in the inverted state most of theacquisition time, the PD’s and memory node dark current anddark signal nonuniformity (DSNU) were improved.
Mid-level pulsed parallel operation of the TX and DCG
gates during charge acquisition provided also additional
antiblooming protection.
During the readout phase, acquired charge in the memory
node was readout in double-sampling mode [first sampleand-hold signal (SHS) and first sample-and-hold reset (SHR)]
followed by readout of charge in both PDs using HCG (second

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
VELICHKO et al.: AUTOMOTIVE 3 μm HDR IMAGE SENSOR WITH LFM AND DISTANCE FUNCTIONALITY

3

Fig. 7. Flickering LED and motion artifacts image comparison: (a) SE
HDR LFM versus (b) three-exposure HDR.

Fig. 8. pLFM pixel timing and potential diagrams.

Fig. 6. (a) SE 120 dB dynamic range total SNR versus temperature for
16-ms integration time and (b) pixel HDR LFM signal linearity versus lux
illuminance.

SHR and second SHS) and LCG (first SHR and third SHS)
correlated double-sampling techniques.
The total SNR comparison of room versus +100 ◦ C junction
temperature in Fig. 6(a) reflected sensor’s 120 dB HDR LFM
performance.
This mode of operation also provided for excellent image
quality with all total SNR transitions at or above 30 dB for+100 ◦ C and 16-ms integration time [Fig. 6(a)] and even for+125 ◦ C and shorter 8-ms integration time. High performanceat elevated temperatures achieved via further process andpixel operation optimization controlling the memory node darkcurrent and DSNU that changed significantly between roomand +125 ◦ C temperatures [9], [10]. The first SNR transition
happened between readout of PDs signal in the HCG modeand readout in the LCG mode due to higher LCG readout
noise, and the second SNR transition had added pixel memory
dark current and DSNU that were temperature and integration
time dependent. While the readout noise of double-sampling
readout caused some reduction of SNR in second transition
at room temperature, but it was not a significant factor athigh temperature where memory node dark current and DSNU

were dominant. The third SNR transition was between first andvery short second exposure. The ratio between first and second
exposures controlled how deep the SNR reduction could be.
Thus, the third SNR transition was not temperature-dependent.
Linearity measurements from 0.01 lx to almost 200 K lux
showed excellent color reproducibility by staying within +1%
to −1.5% of signal, from less than 0.5 lx to max level inFig. 6(b). Pixel’s SE HDR LFM operation captured brightflickering LEDs and eliminated motion artifacts in comparison
to classical multiexposure HDR [Fig. 7(a) versus (b)].
The pLFM method that we developed earlier [3] and implemented in the sensor allowed capturing flickering light sourcesof any brightness up to millions of lux. The pLFM operation
timing diagram is shown in Fig. 8.
In Fig. 8, TXA/TXB and SGA/SGB gates are fully pulsed
during charge acquisition in a push–pull manner [3]. Their
duty cycles regulate electronic PD responsivity in a wide range
as well as charge summing accumulation in the memory node.
DCG gate may be pulsed as shown to allow capacitor C
to connect to the overflow path via the FD and thus extend
dynamic range of the captured pLFM image. DCG gate may
also be asserted for the entire length of the charge acquisition.
Different pulse timing of TXA/SGA and TXB/SGB pairs created much higher sampling frequency and thus better capturingof very low duty cycle LED pulses that could be captured atleast by one PDA or PDB.

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4

IEEE TRANSACTIONS ON ELECTRON DEVICES

Fig. 9. Flickering LED turn lights, headlights, and brake lights image
captures three-exposure HDR (top) versus pLFM HDR (bottom).

Fig. 11. Pixel array geometrical optics construction for phase detect.

Fig. 10. Phase detect pixel with two PDs and its FDTD optical simulation
versus light angles.
Fig. 12. Different response in the phase detect pixels PDs.

During the readout phase, acquired charge in the memory
node was readout in double-sampling mode (first SHS and first
SHR) followed by HCG correlated double-sampling readout(second RST and second SHS) of the remaining charge inboth PDs.
Image comparison of capturing flickering LED turn lights,
headlights, and brake lights using three-exposure HDR andpLFM are shown correspondingly in the top and bottom
images of Fig. 9. These images confirmed effective capturingof automotive LED light sources in true color.
V. D ISTANCE AND I MAGE S IMULTANEOUS ACQUISITION
Two-PD pixels in the sensor covered by a single microlens
additionally enabled distance extraction along with a color
image using in-pixel phase detect [11]. This was first application of two-PD in-pixel phase detect to HDR pixel architecture.
Fig. 10 shows a finite-difference time-domain (FDTD) opticalsimulation of the pixel with two PDs in relation to angles ofgreen light between +30◦ and −30◦ .

Objects at different distances from the imaging lens generate
differences in the signal acquired by each of two PDs within
the pixel.
Fig. 11 shows the principle of the phase mechanism using
a geometrical optics construction. An object in perfect focus
would form a perfect image point on the sensor plane producing zero phase. When an object is out of focus, its image point
is beyond the sensor plane resulting in a cone of rays forming
the circle of confusion at the sensor plane. Depending on the
degree of defocus, the circle of confusion may span several
pixels of the sensor array.
At each pixel PD location, the angle of the incoming
ray is sensed by the phase-detect auto-focus (PDAF) sensor
producing a phase response (Fig. 12). The following derivationconsiders the image of a point source, but in practice, phasemeasurements are derived from the analysis of edges in thescene.
Fig. 13 shows the basic principles of phase generation thatcan be derived from geometrical optics. For the derivation

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
VELICHKO et al.: AUTOMOTIVE 3 μm HDR IMAGE SENSOR WITH LFM AND DISTANCE FUNCTIONALITY

5

Fig. 14. Pixel depth range versus lens f-number and focal length.

Fig. 15. Captured color image and corresponding signal phase/distance(close objects are reddish and distant objects are bluish).
Fig. 13. Geometrical derivation of near defocus response.

of the phase displacement, we consider a lens focused on
an object at a distance S0 . This places the image plane at a
distance of S0 . A point source closer to the lens will be out offocus creating a blur circle at the focal plane of the imager. For
simplicity, we assume that all pixels within the blur circle areuniformly illuminated and those outside receive no photons,
although in reality, diffraction effects will cause the blur circle
to have some Gaussian-like intensity distribution.
D is one half of the clear aperture of the lens. By definitionof the f -number of a lens, N is equal to the focal length overthe clear aperture diameter. By similar triangles, the maximum
extent X max of the blur circle may be expressed asf (s  − s0 )
.
(1)
2Ns 
For any point on the sensor array across the extent of theblur circle, we can calculate the angle of the light ray froman out of focus point as follows:


x
−1
.
(2)
θ = tans  − s0
X max =

Based on the above derivation in (1) and (2), we conclude
that both the extent of the optical spot and the angles of theincoming rays are dictated by the diameter of the entrance
pupil, which is directly related to the lens f -number and thelens focal length, and thus, the depth capabilities are directly
analogous to the depth of field characteristics of the lens.
Wide-angle lenses with large depth of field in which all objects
are effectively in focus provide very little phase informationand thus have limited depth range. Telephoto lenses withlonger focal lengths and lower f -numbers will have shallower
depth of field allowing adequate phase information to bemeasured from more distant objects. However, there is apractical limit where near objects become too blurred tobe adequately measured. Fig. 14 shows the pixel distance
performance characteristics for different lens f -numbers andfocal length.
Two-PD implementation of same-color pixels under a single
microlens allowed effective distance extraction using edgesof different objects in sceneries. Both color HDR image anddistance information of up to a few meters could be acquiredwith reasonable accuracy as shown in Fig. 15, where rightimage distance data red color showed objects closer and blue
color showed objects farther from the camera.
This additional feature is valuable for some automotive incabin and close proximity applications that require fusion of
distance and image data.

VI. P ERFORMANCE C OMPARISONS
Some of the latest HDR LFM backside-illuminated (BSI)
pixel performance metrics are summarized in Table I. In thetable, different architecture pixel types were captured as well
as some of their major parameters: pixel linear full-well (LFW)
capacity, maximum dynamic range, LFM HDR, maximum
SNR in transitions versus temperature, and available modesof operation.

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
6

IEEE TRANSACTIONS ON ELECTRON DEVICESTABLE IM AJOR PARAMETER C OMPARISON OF L ATEST HDR LFM P IXELS

Additional mode of operation in conjunction with two PDs
covered by a single microlens enabled up to few meter distance
extraction along with a color image using in-pixel phase detectmechanisms.
Our future work is focused on smaller and better performing
HDR LFM pixels that are based on the same architecturalprincipals.
ACKNOWLEDGMENT
The authors would like to thank Ross Jatou, Chris Adams,
and Rick Mauritzson for support and proofreading.
R EFERENCESIn Table I, some of the parameters were extracted from plots
or deduced using other parameters. Symbols “∼” and “<”were used in these cases.
VII. C ONCLUSION
We discussed the performance and features of the 1.3-MP
automotive HDR image sensor based on enhanced 3 μm pixel
with two PDs and eight gates. The pixel architecture allowed
the implementation of multiple operation modes, including
multiexposure HDR, SE LOFIC HDR LFM, and pLFM. These
modes covered entire range of typical automotive sceneries
and allowed effective mitigation of flickering LEDs and lights.
Dynamic range achieved up to 140 dB with all total SNR
transitions at or above 30 dB even for greater than +100 ◦ Cjunction temperatures. In combination with HCG, low lightsignal readout that provided for a cinematographic image
quality with sharp high color fidelity captures due to linearity
between +1% and −1.5% of signal from very low to very
bright light levels.

[1] J. Solhusvik et al., “A 1280×960 3.75 μm pixel CMOS imager withtriple exposure HDR,” in Proc. Int. Image Sensor Workshop, Jun. 2009,
pp. 344–347.
[2] S. Velichko et al., “140 dB dynamic range sub-electron noise floor image
sensor,” in Proc. Int. Image Sensor Workshop, Jun. 2017, pp. 294–297.
[3] C. Silsby, S. Velichko, S. Johnson, Y. P. Lim, R. Mentzer, and J. Beck,“A 1.2 MP 1/3’ CMOS image sensor with light flicker mitigation,” inProc. Int. Image Sensor Workshop, Jun. 2015, pp. 405–408.
[4] S. Iida et al., “A 0.68e-rms random-noise 121 dB dynamic-range subpixel architecture CMOS image sensor with LED flicker mitigation,” in
IEDM Tech. Dig., Dec. 2018, pp. 2–10.
[5] Y. Sakano et al., “A 132 dB single-exposure-dynamic-range CMOS
image sensor with high temperature tolerance,” in IEEE Int. Solid-State
Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2020, pp. 105–107.
[6] J. Solhusvik et al., “A 1280×960 2.8 μm HDR CIS with DCG and splitpixel combined,” in Proc. Int. Image Sensor Workshop (IISW), 2019,
pp. 254–257.
[7] S. Sugawa et al., “A 100 dB dynamic range CMOS image sensor using alateral overflow integration capacitor,” in IEEE Int. Solid-State Circuits
Conf. (ISSCC) Dig. Tech. Papers, Feb. 2005, pp. 352–353.
[8] Y. Sakano et al., “224-ke saturation signal global shutter CMOS
image sensor with in-pixel pinned storage and lateral overflow
integration capacitor,” in Proc. Symp. VLSI Circuits, Jun. 2017,
pp. C250–C251.
[9] M. Oh et al., “3.0 μm backside illuminated, lateral overflow, highdynamic range, LED flicker mitigation image sensor,” in Proc. Int. Image
Sensor Workshop, Jun. 2019, pp. 262–265.
[10] M. Oh et al., “Automotive 3.0 μm pixel high dynamic range sensor with
LED flicker mitigation,” Sensors, vol. 20, no. 5, p. 1390, 2020.
[11] S. Choi et al., “An all pixel PDAF CMOS image sensor with
0.64 μmx1.28 μm photodiode separated by self-aligned in-pixel deep
trench isolation for high AF performance,” in Proc. Symp. VLSI Technol.,Jun. 2017, pp. T104–T105.

Authorized licensed use limited to: ON Semiconductor Inc. Downloaded on April 26,2022 at 20:25:01 UTC from IEEE Xplore. Restrictions apply.

onsemi,, and other names, marks, and brands are registered and/or common law trademarks of Semiconductor Components Industries, LLC dba “onsemi” or its affiliates
and/or subsidiaries in the United States and/or other countries. onsemi owns the rights to a number of patents, trademarks, copyrights, trade secrets, and other intellectual property.
A listing of onsemi’s product/patent coverage may be accessed at www.onsemi.com/site/pdf/Patent−Marking.pdf. onsemi reserves the right to make changes at any time to any
products or information herein, without notice. The information herein is provided “as−is” and onsemi makes no warranty, representation or guarantee regarding the accuracy of theinformation, product features, availability, functionality, or suitability of its products for any particular purpose, nor does onsemi assume any liability arising out of the application or useof any product or circuit, and specifically disclaims any and all liability, including without limitation special, consequential or incidental damages. Buyer is responsible for its products
and applications using onsemi products, including compliance with all laws, regulations and safety requirements or standards, regardless of any support or applications information
provided by onsemi. “Typical” parameters which may be provided in onsemi data sheets and/or specifications can and do vary in different applications and actual performance mayvary over time. All operating parameters, including “Typicals” must be validated for each customer application by customer’s technical experts. onsemi does not convey any licenseunder any of its intellectual property rights nor the rights of others. onsemi products are not designed, intended, or authorized for use as a critical component in life support systems
or any FDA Class 3 medical devices or medical devices with a same or similar classification in a foreign jurisdiction or any devices intended for implantation in the human body. Should
Buyer purchase or use onsemi products for any such unintended or unauthorized application, Buyer shall indemnify and hold onsemi and its officers, employees, subsidiaries, affiliates,and distributors harmless against all claims, costs, damages, and expenses, and reasonable attorney fees arising out of, directly or indirectly, any claim of personal injury or deathassociated with such unintended or unauthorized use, even if such claim alleges that onsemi was negligent regarding the design or manufacture of the part. onsemi is an Equal
Opportunity/Affirmative Action Employer. This literature is subject to all applicable copyright laws and is not for resale in any manner.

PUBLICATION ORDERING INFORMATION
LITERATURE FULFILLMENT:
Email Requests to: orderlit@onsemi.com
onsemi Website: www.onsemi.com◊

TECHNICAL SUPPORTNorth American Technical Support:
Voice Mail: 1 800−282−9855 Toll Free USA/Canada
Phone: 011 421 33 790 2910

Europe, Middle East and Africa Technical Support:
Phone: 00421 33 790 2910
For additional information, please contact your local Sales Representative