The 70th Japan Society of Applied Physics Spring Meeting 2023
Highlighted Presentations Press Release
April 18, 2023

Dielectric Metasurfaces for High-sensitivity, Multi-dimensional Imaging

Dielectric metasurfaces realize
Image sensing more than human eyes
NTT Device Technology Laboratories
Masashi MiyataSummary• Dielectric metasurfaces enable high-sensitivity image sensors with light-receiving
capacity about three times greater than existing image sensing.
• By combining Metalens, a unique optical technology based on dielectric metasurfaces,
and cutting-edge image-processing technology using AI, Masashi Miyata and his researchgroup at NTT Device Technology Laboratories have developed an imaging technology
capable of acquiring hyperspectral images.
• This technology can be implemented in existing smartphones and compact digital
cameras and can democratize advanced image sensing using hyperspectral imaging.
Masashi Miyata and his research group at NTT Device Technology Laboratories have developed an imaging technologythat can acquire hyperspectral images by combining a Metalens technology and cutting-edge image processing
technology using AI. Another feature of this technology is that it works by simply replacing the lens of an existingsmartphone or compact digital camera with a Metalens. Hyperspectral images can identify the properties of objects thatare difficult for a human eye to grasp and are expected to open new possibilities in industries such as agriculture,
forestry, fishery, healthcare, and smart manufacturing.

Detail

The world is filled with light invisible to human eye
The word "imaging" generally evokes images that a human eye can see, that is, images created byvisible light. However, many things in this world cannot be understood solely through images
captured by the human eye. For example, imaging using light other than visible light, such as
observing the body using X-rays and the analysis of crystal structures using infrared spectroscopy,
are indispensable for medical and scientific progress. Imaging sensors that capture light invisible tohumans have potential applications such as smartphones, automated driving, and wearabledevices. Strangely, our society is progressing through technology that captures light that we cannot
see.
"Currently available smartphone and general cameras are designed to capture images that areoptimized for the human eye, which means that their image sensors are limited to capturing only
the wavelengths of light visible to humans. The motivation behind our research is to fundamentallychange that," explained Masashi Miyata of NTT Device Technology Laboratories.
Miyata and his research group have developed an optical technology based on a unique artificial
surface structure called “Metalens.” Unlike conventional lenses, Metalenses can control andseparate a wide variety of color information with a single lens. When mounted on an image sensor,
they can maximize the amount of light received (2.83 times more than conventional sensors). The
research group has also developed a technology that enables the acquisition of hyperspectral
images(*) by combining this Metalens with state-of-the-art image processing using AI.
Furthermore, they have made this technology applicable to existing smartphones and compact
digital cameras.
High-sensitivity image sensor realized by dielectric metasurface
The base of the Metalens is called a "dielectric metasurface." Dielectric metasurfaces generally
refer to the concept of artificial surface materials based on dielectric surface nanostructures which
are smaller than the wavelength of light and exhibit unique optical properties. Miyata focused on
the potential of dielectric metasurfaces as an optical device technology and led the research atNTT Laboratories.
The dielectric metasurfaces can control light intensity, phase, polarization, and dispersion. By
integrating them into an image sensor, it is possible to achieve higher sensor functionality andperformance. The application of this dielectric metasurface to lenses is called a Metalens. Anotherfeature of this technology is its simplicity because it can be implemented in existing opticalsystems.
Miyata et al. used dielectric metasurfaces to increase the sensitivity of image sensors. The highsensitivity of image sensors is an advantage for industrial applications. For example, highsensitivity image sensors enable object recognition using high-speed imaging at night. This canimprove the accuracy of automatic driving technologies.
In existing technology, there is always a trade-off between higher resolution and higher sensitivity.
For example, if the spatial resolution increases, the pixel size of the image sensor must be finer.
However, miniaturization reduces the amount of light collected per pixel, resulting in decreasedsensitivity. To achieve higher sensitivity, technological innovation is required to significantlyincrease the amount of light received per pixel. Miyata et al. developed a technology that useddielectric metasurfaces for color sorting.
In the existing image sensors, information about the three primary colors of light (red, green, andblue) is acquired using three light-sensitive elements. A color filter is placed in front of eachelement to absorb unnecessary colors and transmit only the necessary colors. Therefore, theamount of light received by the image sensor is approximately one-third of the incident light. Usinga lens based on dielectric metasurfaces for separating different colors from the light, a method for
acquiring color information without using color filters was established .
They fabricated dielectric metasurfaces by etching thin films of silicon nitride. The microstructure
formed by the small pillars that divide the incident light can distribute light into pixels corresponding
to red, green, and blue light (Figure 1). The experimental demonstrations confirmed a 2.83-foldincrease in the amount of light received.
Dielectric metasurfaces can be mass-produced using existing semiconductor processes. Inaddition, a new technology has been developed that allows direct patterning, similar to stamping,using an industrial high-refractive-index resin (made by NTT-AT) and nanoimprinting.

Democratize “more than eyes"
This research group aims to acquire diverse information about real-time environments bycombining dielectric metasurfaces with information technology, sensor technology, andbiomimetics. The most outstanding research area is the development of a technology for acquiring
hyperspectral images and videos using an "ordinary camera" with Metalenses and AI-based
information processing.
When a camera with a Metalens captures an image, the Metalens captures the shape andwavelength information of the object as a compressed image. This image is then reconstructed asa hyperspectral image through image processing, using an AI algorithm based on a neural network(Figure 2).
"The main feature of this technology is that it can realize hyperspectral imaging with an ordinary
digital camera without sacrificing the fundamental imaging performance. This is achieved using aneural network originally developed by NTT Laboratories to reconstruct a large amount ofinformation back from a small amount of information," said Miyata.
The current research results show that hyperspectral images have been obtained at 45
wavelengths, ranging from visible light to near-infrared light, and the same is true for videos (HDresolution and frame rate of 30 fps). This camera architecture can be installed on existingsmartphones and compact digital cameras, making it a technology that can democratize "eyesbeyond the human eye."Based on previous research, theyare also considering applications such as analyzing bloodvessels, food, and agricultural products from captured hypersptectral images. "In future, we willconsider using this technology for determining the state of a person's health by taking a picturewith a smartphone. We may even be able to tell if a tomato tastes better by taking a photo of it at afarm,” said Miyata.

Annotation*Hyperspectral image: A hyperspectral image is an image (group of images) captured by spectroscopy to obtain color
information using more wavelengths than a typical digital camera. Generally, a hyperspectral image is not a single image.
It is a group of several dozen images, each of which is represented by multiple images based on color information atdifferent wavelengths.
FiguresFigure 1. Color Sorting by Dielectric Metasurface

Figure 2. Hyperspectral imaging with Metalens