Latest Trends and Future Directions in Ultra-High-Definition
Imaging Technology Accelerating Smart Industry

Masatsugu Onizuka

Imaging System Business Division
IS Application Engineering Department,
Sony Semiconductor Solutions Corporation

1. Introduction

4K and 8K ultra-high definition imaging technology is being
used to improve image quality in broadcasting, but it is also
being used in various other fields, such as to display advertisingon outdoor displays, to implement ultra-high-definition systems
for security and crime prevention, and for medical applications.
In industry, where smart technologies are being used to increase
productivity dramatically as we enter the IoT age, high-definitionand high-quality images can be taken, and there is increasing useof sensing technologies utilizing infrared and other images not
visible to the human eye. Image sensors are a key type of device
necessary for these advanced types of machine-vision camera.
Industrial image sensors require high quality and continuousavailability. Industrial applications vary widely, so a significant
feature of this market is the need for a broad product lineup. We
are a leading company in image sensors, with a broad portfolio of
products covering small to large, low to high-speed, low to highpixel counts, and from ultra-violet through infrared frequencies, tomeet all kinds of requirements. Each series is also packaged withuniform pin-outs, so it is easy to develop product lineups whendeveloping machine vision cameras.
This article describes our vision for advancement of image
sensors for industrial applications and introduces some examples ofpractical applications. Figure.1 shows our vision for the evolutionof image sensors in the future. We have defined three directionsfor advancement. The first is to improve the performance of ourbase image sensors, the second is to extend performance fromimaging to sensing, and the third is to optimize functionality foredge systems. We describe each of these perspectives in detailbelow, and also give examples of related machine vision camerassystems.
■ Figure 1

18

New Breeze Winter 2023

2. Advances in imaging performance

To implement advanced automation in areas such as factories
or logistics requires improved efficiency by increasing theaccuracy and decreasing the time required for inspections. Formachine vision cameras, which handle most of these functions,this requires image sensors with higher imaging performance interms of resolution (higher definition), higher speed, and otherfactors. This section examines typical image sensor performance
improvements in the areas of increasing resolution while reducing
size, and technologies to improve productivity by increasing
image-sensor size while accelerating data read-out.
2.1 Achieving both higher resolution and smaller size
When attempting to improve accuracy for visual processes
detecting small defects or contamination, simply increasing thepixels in the image sensor results in larger chip size and larger
cameras, which can be an issue. Conversely, if the pixels are
made smaller to avoid increasing the camera size, the lightcollecting area per pixel decreases, leading to decreased image
quality and unavoidable decreases in recognition and detection
performance. To resolve this issue, we have developed a layered
CMOS image sensor technology called Pregius STM (Figure 2, 3)
that is suitable for industrial applications. It has a “Global Shutter”
feature that uses a back-illuminated pixel structure to achieving
both high resolution and smaller size. The new structureis able to collect more light per pixel than conventional frontilluminated structures, and we developed technology to mask thememory components in our global shutter function, so that pixel
characteristics are not compromised, so we were able to reduce
pixel size to 2.74 μm diagonally, or 63% of earlier pixels. By■ Figure 2

■ Figure 3■ Figure 4■ Figure 5

layering the signal processing circuits, which were conventionallyplaced in beside the pixels, we have also increased resolution byapproximately 1.7 times for the same optical system (from 12.37
M pixels to 2.035 M pixels) and reduced the package size (to 91%of earlier*1 ) (Figure 4).

■ Figure 6

2.2	Image sensor technologies for larger size and fasterreadout
Some machine vision cameras use an image sensor that
supports ordinary C-mount lenses, but further productivity
increases can be obtained by using an image sensor with a larger
optical size to increase the imaging area. For example, comparing
our IMX235 image sensor, which is 1.1-type (diag. 17.6 mm)
and has 12.37 M pixels, and our large-aperture IMX661 image
sensor, which has our Global Shutter function, is 3.6-type (diag.
56.7 mm) and has 127 M pixels, productivity can be increased
by reducing the number of images captured, and recognition
accuracy can be improved with the higher resolution (Figure 5).
For inspection of flat panel displays, if the image sensor resolution
is too low relative to the resolution of the panel, moiré patterns
occur as shown in Figure 6, so inspection with oversampling
using an ultra-high-definition image sensor is very useful. On theother hand, when the number of pixels is increased, generally theamount of signal processing required also increases, the frame rate
decreases and more time is required to read out the data. With
the IMX661, we used a Chip-on-Wafer process technology tocreate an original device structure, layering a chip with some ofthe functionality onto the pixel wafer and increasing processing
performance by implementing AD converters in an optimized
location. We also used a high-speed interface called Scalable Low
Voltage Signaling with Embedded Clock (SLVS-EC) to achievea four-times increase in image read-out speed* . For inspection＊1 Comparing our IMX531 and IMX253 CMOS image sensors with the Global Shutter function. “Pregius S” is a trademark or registered trademark of the Sony Group and related companies.

New Breeze Winter 2023

19

of devices such as flat-panel displays and printed circuit boards,
cameras with large-aperture image sensors are being used more
and more. Larger areas to be imaged in a single shot by usingof larger image sensors and fast read-out technology, reducing
both the number of images taken and the inspection time, andinspection accuracy can be improved by increasing resolution.

3. Extending capabilities to sensing

Recently, the use of sensing beyond the capabilities of humanvision is advancing in the field of industrial machinery. In addition
to conventional inspection and recognition using visible light, newinformation such as near-infrared, polarized light, and distancedata is being captured, making it possible to solve problems
that could not be solved earlier with sensing. In this section, we
describe four of these various new sensing technologies: ShortWavelength InfraRed (SWIR) image sensors, Ultraviolet (UV)light image sensors, Time-of-Flight (ToF) range-imaging sensors,and polarized-light image sensors.
3.1 SWIR Image sensors
SWIR is a type of infrared light with relatively shortwavelengths. In 2020, we announced our IMX990/IMX991
image sensors, able to capture images over a wide range ofwavelengths that include visible light, from 0.4 μm to 1.7 μm.
When developing SWIR sensors, we formed photodiodes in alayer of the compound semiconductor, Indium-Gallium-Arsenide(InGaAs) and the read-out circuits in a silicon layer, making
copper-copper (Cu-Cu) connections between them. This enabled■ Figure 7■ Figure 8＊2 Comparing our IMX661 and IMX253 sensors with the Global Shutter feature.

20

New Breeze Winter 2023

us to reduce the pixel pitch, achieve wide bandwidth and create animage sensor supporting SWIR, which was not possible earlier.
SWIR wavelengths can be used to visualize damage to fruit
below the surface (visualizing differences in moisture density) as
shown in Figure 7, or to select materials utilizing differences inabsorption of short-wavelength infrared light. The new sensor
can be used as a more versatile inspection device to reduce systemcosts, by using a single device for inspection that previously
required separate cameras for visible and SWIR light, or toincrease throughput, by accelerating image processing.
3.2 UV image sensors
Sensing with ultra-violet (UV) light can be useful for selecting
materials difficult to differentiate with visible light, or to detect
fine scratches or defects. UV light has wavelengths that are shorter
than visible light, generally in the range of 10 nm to 400 nm(Figure 8). Our IMX487 image sensor detects wavelengths from
200 nm to 400 nm, and is suitable for industrial sensing using
UV wavelengths. Used with a UV light source, it shows promise
for a wide range of use cases, including detecting semiconductor
pattern defects, differentiating materials for recycling (Figure 9),
and detecting minute scratches on the surface of parts. This
product has a structure specialized for sensing UV wavelengths,
and is equipped with our Pregius S technology, so it produceshigh frame-rate images without motion distortion. We expect it toexpand the range of applications for UV image sensors, including
those requiring high speed.

■ Figure 9■ Figure 10■ Figure 11

3.3 Polarized light image sensors
Normally, when looking at an object, we see both light that isreflected and light that is diffused from the surface of the object.
This light oscillates in different directions and it is possible toselect light oscillating in a particular direction (polarization) by
passing it through a polarizer (Figure 10). Our polarizing imagesensors (e.g.: IMX253MZR) are equipped with internal polarizers
for four directions and are able to capture polarization data forthese four directions in a single image (Figure 11). As shown in
Figure 12, this polarized light data can be used to increase theaccuracy when detecting scratches, impurities or distortion on thesurface of objects or to eliminate reflected light, which are difficult
using conventional visible-light sensing,. We expect this to enable
a range of other applications in the future.
3.4 Time of Flight depth-image sensors
A Time-of-Flight (ToF) depth-image sensor uses light from alaser or LED to measure the distance to the object, by measuring
the differences in time required for the reflected light to reachthe sensor. We announced the IMX570 image sensor in 2021, as
a compact, high-resolution sensor capable of sensing in 3D space(Figure 13). It was developed for applications that are difficult toimplement with conventional 2D images, such as those requiring
detection of volume or shape, or overlapping of objects.
By providing this lineup of various sensing devices, we arecontributing to solving problems in manufacturing settings suchas eliminating failure, increasing throughput, eliminating line
stoppage, reducing labor requirements and automation.

■ Figure 12

New Breeze Winter 2023

21

■ Figure 13■ Figure 14

4. Functional extensions optimized for edge
systems■ Figure 15

Conventionally, image sensors have been developed with theassumption that human eyes would be looking at the images,
and sensors would output images in terms of effective pixels.
However, when used by AI or other machines, all of the pixel data
will not necessarily be required. For example, when inspecting
printed circuit boards, it may be enough to inspect only specificcomponents or locations. In such cases, pixel data from otherlocations is not needed and capturing the extra data increases
the time required from image capture till peripheral devices
can take action. Even when considering the simple imaging
devices described in earlier sections, the amount and types ofdata is increasing greatly, with more pixels, faster speeds, andadditional data such as polarization and distance. As inspectionof components advances and camera resolution increases, thevolume of information will increase dramatically, also increasing
the processing load for image processing and interface circuits atlater stages.
As such, when considering AI and other machine processing,it will be useful to extract only the regions needed, to reduce theamount of data and the required processing time. Our image
sensors have a Region of Interest (ROI) function to extract only
the area needed, and a self-triggering function to output only datafrom the instant required. They are also equipped with two AD
converters in a layered implementation, which can read data at twodifferent levels of gain, and they have functions to synthesize thesevalues internally. High Dynamic Range (HDR) processing, which
is normally implemented by overlapping multiple images, can beperformed by synthesizing the images within the sensor, without
producing artifacts, and we have implemented highly-robust
sensors able to output the same amount of data as earlier sensors(Figure 14).
This sort of extended functionality can be implemented using
layering technology. As shown in Figure 15, a key point regarding
the layering technology is that the wafer process is separated intoa pixel region and a circuit region. This makes further imagequality improvements and functional enhancements both morescalable. Techniques applied to the wafer process can also beextremely useful in reducing latency, implementing feedback to22

New Breeze Winter 2023

pixels and reducing power consumption. Extending functionality
will surely be helpful as AI is used more and more in the future.
It will be extremely important how this overall issue is handled,including later-stage system processing, and we are continuing towork on both hardware and software solutions for these issues.

5. Conclusion

This article has introduced directions for advancementof image sensors (advancing and extending performance andfunctionality) as considered by Sony Semiconductor Solutions.
We have over 120 types of products developed along thesedirections, to meet a wide range of requirements in industry. We
will continue advancing innovation in image sensor technologies,
cultivating new applications, accelerating smart industry, andcontributing to improving lifestyles and solving societal issues. For
product details and inquiries regarding image sensors for industrial
equipment, please see the following address.
https://www.sony-semicon.com/en/products/is/industry/index.html