3211

Sensors and Materials, Vol. 35, No. 9 (2023) 3211–3225
MYU Tokyo
S & M 3386

Development of 4-μm-pixel Pitch
NIR Multispectral Imaging Sensor and Its Application
to Glare-free NIR Color Fundus Camera
Motoshi Sobue,1,2* Hironari Takehara,1 Makito Haruta,1 Hiroyuki Tashiro,1,3
Kiyotaka Sasagawa,1 Ryo Kawasaki,4 and Jun Ohta1
1Nara Institute of Science and Technology, Division of Materials Science,

8916-5 Takayama, Ikoma, Nara 630-0192, Japan

2Nanolux Co. Ltd., 8F Shinagawa Ground Central Tower, 2-16-4 Konan, Minato-ku, Tokyo 108-0075, Japan
3Kyushu University, Faculty of Medical Sciences, 3-1-1 Maidashi, Higashi-ku, Fukuoka 812-8582, Japan

4Osaka University, Division of Public Health, Department of Social Medicine, Graduate School of Medicine,

2-2 Yamadaoka, Suita, Osaka 565-0871, Japan(Received February 18, 2023; accepted March 20, 2023)

Keywords: electrical biosensors, fundus camera, interference filter, IR color, multispectral imaging
sensorIn this study, we describe the development of a compact NIR multispectral imaging sensor
for use in glare-free NIR color fundus cameras. Integrating NIR technology into a fundus
camera offers significant advantages over conventional RGB imaging using visible illumination,
as it enables the glare-free capture of fundus images with minimal patient discomfort. The
specifications necessary for a glare-free NIR color fundus camera were evaluated on the basis offactors such as pixel pitch size, pixel array layout, and multilayer interference filter design, in
accordance with the camera’s intended purpose. While multilayer interference filters were
deposited on a glass substrate and bonded with the sensor chip in a 7.5 μm pixel pitch in aprevious study, we propose an NIR multispectral imaging sensor directly depositing theinterference filters on the wafer in a narrower 4 μm pixel pitch. In addition, the fabrication
process for directly depositing NIR multispectral filters on the sensor wafer was proposed. The
fabricated NIR multispectral imaging sensor was analyzed against the intended design. Finally,an NIR multispectral imaging sensor was installed on a glare-free NIR color fundus camera, andit was confirmed that the resulting camera is capable of providing medically relevant andmeaningful information.

1.

Introduction

Fundus images contain extensive health information, comprising tissues where the vascularsystem can be directly observed.(1–5) However, conventional visible light color fundus cameras
use intense visible light flash illumination during imaging, which causes discomfort to patients.
In addition, the images must be re-taken in case of an imaging failure. Moreover, pupil shrinkage*Corresponding author: e-mail: sobue.motoshi.sk2@ms.naist.jp

https://doi.org/10.18494/SAM4358
ISSN 0914-4935 © MYU K.K.
https://myukk.org/

3212

Sensors and Materials, Vol. 35, No. 9 (2023)

after the flashlight exposure increases the fundus imaging difficulty. In contrast, an NIR color
fundus camera uses invisible NIR illumination, so patients do not feel glare. Thus, video
recording can be performed, allowing the user to select the best images from the stored video
images. The “glare-free” feature makes it easy to take video images and permits selfie fundus
imaging, which is a significant advantage, especially in home healthcare. A glare-free NIR color
fundus camera can detect vascular morphological changes associated with lifestyle-relateddiseases at an early stage.(6) The availability of non-invasive descriptions of vascular observations
associated with high blood pressure, arteriosclerosis, obesity, and smoking, as well as noninvasive censoring methods to estimate blood concentrations of sugar and fat, enables the early
detection of lifestyle-related diseases.(7) Early detection contributes to the risk stratification andvisualization of lifestyle habit improvements and therapeutic effects. Thus, a glare-free NIR
color fundus camera has excellent potential for use as a personal healthcare device and has broad
applications for ophthalmologists and physicians, medical examinations, general practitioners,
and patients using small inspection instruments in remote diagnoses. Making it compact andaffordable is the key challenge as a “fundus camera anyone can use.”(8–10) In a previous study,(11)it was identified that a single-chip IR multispectral image sensor, including an array of filtersthat transmit NIR light of different wavelengths to each pixel, is critical for providing a compact
and affordable camera system. The previous study proposed an imaging sensor fabricated bybonding a glass substrate mosaic filter to an image sensor at a pixel pitch of 7.5 μm. However, aglass substrate bonding with a sensor chip is unsuitable for affordable and large-scale production.
Moreover, the 7.5 μm pixel pitch is too large to be installed in a compact camera with therequired resolution. In this study, we propose an NIR multispectral imaging sensor that directly
deposits interference filters on a sensor wafer with a narrower pixel pitch of 4 μm. The proposed
NIR multispectral sensor was installed in a glare-free NIR color fundus camera, and its fundus
images were verified to deliver medically relevant and meaningful information. This researchfollowed the Ethical Guidelines for Medical and Health Research Involving Human Subjects at
the Nara Institute of Science and Technology.

2.

Materials and Methods

2.1

Image sensor requirements and specifications

2.1.1 Base sensor selection
We selected a 1/2.6 inch sensor in consideration of compactness and affordability among thecommercially available imaging sensors because an approximately 1/3 inch sensor is prevalent
in a large surveillance market and is relatively low in price. The product name of the selected
sensor is HM5533 manufactured by Himax Technologies, Inc. The selected sensor has a pixel
pitch of 2 μm and a back-side-illuminated design. We considered two types of sensor candidates
in the same format, namely, a standard sensor with a flat surface and an IR-enhanced sensor
with an inverted pyramid array (IPA). We selected the standard type because the IR-enhanced
type is more likely to cause cross-talk between pixels owing to an IPA surface in narrow pixel

Sensors and Materials, Vol. 35, No. 9 (2023)

3213

pitches, resulting in inaccurate wavelength resolution.(12,13) This is particularly concerning
because NIR light penetrates deep into silicon and generates photocarriers reaching theneighboring pixels,(14) whereas the selected sensor does not have mechanical solutions preventing
cross-talk, for example, deep trenches between the pixels. Although a standard sensor may notexhibit the same level of IR sensitivity as an IR-enhanced sensor, particularly beyond 900 nm,
we have prioritized wavelength resolution to attain the IR multispectral advantage. To counteract
the low-sensitivity problems, as outlined in Sect. 2.1.3, we have employed a pixel array layout
strategy. The specifications and relative quantum efficiency (QE) of the selected sensor are listed
in Table 1 and Fig. 1, respectively.
2.1.2 Pixel binning optimization
We analyzed the optimal pixel pitch for this study because there is a trade-off between thespatial and wavelength resolutions based on the pixel pitch size. The spatial resolution is higher
for smaller pixel pitches than for larger ones. However, the wavelength resolution is worse forsmaller pixel pitches than that for larger ones, particularly in NIR, owing to greater interference
between adjacent pixels. This interference is more significant in smaller pixels because the ratio
Table 1
Specifications of the selected sensor.
Item
Sensitivity
Sensor size
Number of pixels
Pixel size
DesignShutterMaximum frame rate
Dynamic range
S/N ratio (max)Video interface
Output format
Power consumption (MIPI 4-lane 45 FPS)Power consumption (MIPI 2-lane 30 FPS)

Description
1402 mV/Lux-sec
1/2.6 inch
5.5 mega (2720 × 2080)
2.0 × 2.0 µm 2
Back-side-illuminated design
Rolling shutter
30 FPS (2-lane MIPI), 45 FPS (4-lane MIPI)
120 dB
37.33 dB
MIPI-CSI2, 1 Gbps per lane
10-bit RAW
171 mW (typ.)145 mW (typ.)*Noted that the data was quoted from the manufacturer’s datasheet.

Fig. 1.

(Color online) Relative QE of selected sensor without color filters.

3214

Sensors and Materials, Vol. 35, No. 9 (2023)of the length of the four sides to the overall area of a pixel is higher than that of larger pixels,
resulting in a higher chance of cross-talk and a lower sensitivity of the targeted wavelength. In
this study, given that the physical pixel size of the selected sensor is a pixel pitch of 2 μm, thealternatives for the pixel binning and effective pixel pitch include (a) single-pixel binning at 2
μm, (b) quad binning at 4 μm, and (c) 9-folded binning at a pixel pitch of 6 μm (Fig. 2).
To analyze optimal pixel binning, we first analyzed the spatial resolution requirements of thisstudy. One of the critical requirements for the early detection of lifestyle-related diseases is themeasurement of the relative thickness of the blood vessels. It is expected to have 10 pixels within
150-μm-diameter blood vessels in the fundus image to observe the shape of the blood vessels.
Because the viewing angle of the fundus image is required to be at least 30 degrees around thecenter of the optic disk with 24 mm of the eye axial length of an average person, the target size ofthe image is calculated as 16.4 × 12.6 mm2. The vertical resolution requirement is 838 pixels,
which was calculated using the image size, target size, and resolution target, as shown in Fig. 3.
The vertical resolutions of the 2, 4, and 6 μm pixels are 2080, 1040, and 693, respectively.

Effective
pixel size

(a)

(b)

(c)

Fig. 2. (Color online) Pixel binning and effective pixel pitch: (a) single-pixel binning and effective pixel pitch of 2
µm, (b) quad binning and effective pixel pitch of 4 µm, and (c) 9-folded binning and effective pixel pitch of 6 µm.

(a) Required angle: 30 degrees around optic disk(b) Eye axial length: 24 mm (normal)(c) Vertical image size: 12.6 mm, from (a) and (b)(d) Horizontal image size: (c) × aspect ratio 1.3 = 16.4 mm(e) Target size: 0.15 mm (150 µm)(f) Resolution target: 10 pixels(g) Vertical resolution requirement: (c)/(e) × (f) = 838 pixels(h) Horizontal resolution requirement: (d)/(e) × (f) = 1096 pixels
Fig. 3.

(Color online) Fundus camera resolution requirements.

Sensors and Materials, Vol. 35, No. 9 (2023)

3215

Therefore, the spatial resolution of the pixel pitch of 6 μm (9-folded binning) does not meet therequired standard, whereas the pixel pitches of 2 and 4 μm satisfy the requirements. Evenconsidering the individual difference in eye axial length ranging from 23 to 26 mm, theanalytical result is consistent as the vertical resolution requirement is in the range of 803–908
pixels, more than the vertical resolution of the pixel pitch of 6 μm (9-folded binning) but lessthan those of the pixel pitches of 2 and 4 μm.
Second, we assessed the wavelength resolutions of the 2, 4, and 6 μm pixel pitches. To
analyze the pixel pitch and spectral performance, the optical flows in the multispectral filters
were simulated using the three-dimensional finite-difference time-domain (FDTD) method,which can simulate various electric and magnetic materials using a personal computer. We built
a multilayer interference filter structure, as discussed in Sect. 2.1.4, by arranging three different
multilayer interference filters (800, 870, and 940 nm) set on the Bayer pattern in the simulation
model. Specific wavelength light (800, 870, or 940 nm) illuminates the filter perpendicularly[Fig. 4(a)]. The horizontal grid size of the FDTD model was set as 51 × 51, 101 × 101, and151 × 151 at pixel pitches of 2, 4, and 6 μm, respectively; the vertical grid size was set as 1529 for
all pixel pitch sizes [Fig. 4(b)], balancing the robustness of the results and the computational time
to convergence. In the simulation, light with a specific wavelength of 800, 870, or 940 nm wasirradiated onto the pixels, and the transmittance of the target filter(s) was measured. The averagetransmittance rates of the two 800 nm filters with 800 nm light irradiation were 55.4, 63.8, and70.6% at pixel pitches of 2, 4, and 6 μm, respectively [Fig. 4(c)]. The transmittance rates at 870
and 940 nm were above 90% for all pixel pitch sizes [Figs. 4(d) and 4(e)]. Our goal was to achievea transmittance rate of 60% for all 800, 870, and 940 nm filters, based on the previous study
where successful fundus images were obtained using IR1, IR2, and IR3 multilayer interference
filters with an average transmittance rate of around 60%.(15) These simulation results showedthat the pixel pitch of 2 μm (single pixel) at the 800 nm filter had a transmittance rate of 55.4%
and did not meet the target (60%), whereas the 4 and 6 μm pixel pitches satisfied therequirements.
In summary, from the spatial and wavelength resolution performance characteristics, only
the pixel pitch of 4 μm (quad binning) meets both the targets. The pixel pitch of 9 μm was toolarge to achieve the spatial resolution target and the pixel pitch of 2 μm was too small to achieve
the wavelength resolution target (Table 2). Thus, we decided to deposit the multispectral filter on
every four pixels through quad binning.
2.1.3 Pixel array layout
The pixel array layout is another important element in image sensor design. Most RGB
sensors have a Bayer arrangement in which four pixels form one unit, with one pixel each for R
and B, and two pixels for G.(16) In the previous study, the pixel array layout was IR1 (800 nm),IR2 (870 nm), and IR3 (940 nm) × 2 in a Bayer arrangement [Fig. 5(a)]. If the same filter
arrangement is applied in this study, the sensitivity will be very low because of the low QE of theselected sensor in the NIR range, particularly above 900 nm. To resolve this issue, a modified
filter arrangement was proposed by replacing two IR3 with two IR1 and shifting the wavelengths

3216

Sensors and Materials, Vol. 35, No. 9 (2023)

(a)

(b)
Fig. 4. (Color online) (a) Pixel array unit layout for three-dimensional simulation. The number on each pixel
indicates the designed peak transmittance wavelength. (b) Calculated transmittance of each pixel for the effective
pixel pitches of 2, 4, and 6 µm for incident light wavelengths of 800, 870, and 940 nm. The transmittances of pixels
with the same wavelength of incident light and the designed peak transmittance wavelength are indicated by red
rectangles.

Table 2
(Color online) Pixel pitches and evaluation results.
Target2 µm
Resolution: vertical pixels
873
2080
Lowest % transmittance (%)
60
55.4*Note that the hatched elements are above the target.

4 µm
1040
63.8

6 µm
693
70.6

Sensors and Materials, Vol. 35, No. 9 (2023)

(a)

(b)

3217

(c)

(d)

Fig. 5. (Color online) Filter array unit arrangements: (a) previous study, (b) this study – IR-color-oriented,(c) alternative 1 – sensitivity-oriented, and (d) alternative 2 – multispectral-oriented.

slightly shorter than that in the previous study to IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm)[Fig. 5(b)]. The new filter arrangement was expected to enhance the total sensor sensitivity by
reducing low-sensitivity wavelengths and increasing high-sensitivity wavelengths. The
sensitivities of the selected sensor are 30, 16, and 8% at IR1 (800 nm), IR2 (870 nm), and IR3(940 nm), respectively, as shown in Fig. 1; the average sensitivity per pixel is 16%. By modifying
the pixel array as proposed, the average sensitivity per pixel will be improved to 25% or 1.6
times more than that in the previous study. Because the wavelength difference is only 20 nm, it
is not considered to significantly impact the NIR color quality. We considered alternative pixel
array layouts, as shown in Figs. 5(c) and 5(d). Figure 5(c) shows a layout aiming for enhanced
sensitivity by shifting wavelengths much shorter than those in Fig. 5(b), and Fig. 5(d) shows alayout aiming for higher spectral resolution with four different multispectral filters. Although
the layouts described in Figs. 5(c) and 5(d) are potentially advantageous, these do not fit thisstudy because the wavelengths shorter than those in Fig. 5(b) are dazzling to human eyes andagainst the idea of a glare-free fundus camera. Therefore, we selected Fig. 5(b) as the pixel arraylayout for this study.
2.1.4 Multilayer interference filter design
The filter designs used to discriminate IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm) wereanalyzed. Although visible RGB Bayer filters use absorption filters composed of pigments
mixed with organic materials, no reliable absorption filter materials lie in the NIR range. In aprevious study, a mosaic filter was fabricated using dielectric multilayer interference filters,
which could be precisely designed using optical simulations, provided that the optical propertiesof the materials were known.(17) Fabry–Perot bandpass filters are significantly advantageous
because a film with a half-wavelength thickness between the two mirrors can invert the reflected
light phase, causing light close to the reference wavelength to be transmitted. Moreover, Fabry–
Perot bandpass filters can control the transmission wavelength by adjusting the thickness of theintermediate layer. Because the thickness adjustment involves only one layer, the fabrication of amosaic-like filter is relatively simple without numerous repetitive etching processes.

3218

Sensors and Materials, Vol. 35, No. 9 (2023)

Multilayer designs of the interference filters are shown in Fig. 6(a). The bottom-half and tophalf layers were common to all three filters. The only difference was the thickness of theintermediate layer. As indicated in Fig. 6(b), the transmission spectra of the filters can achieve
the target wavelengths of IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm) by the design. For the
dielectric materials, we selected TiO2 (a high-refractive-index material) and SiO2 (a lowrefractive-index material), which are commonly used non-metal materials in multilayer thin film
factories. The total thickness of the filters is 1.5 μm, which is sufficiently small to apply to a
pixel pitch of 4 μm.(15) While multilayer interference filters were deposited on a glass substrate
and attached to a sensor chip in the previous study, in this study, the filters were deposited
directly on the sensor wafer. This contributes to a significant cost reduction of the chip byeliminating the glass substrate and simplifying the fabrication process.
2.2

Fabrication process

We proposed a fabrication process for depositing multilayer interference filters onto a sensor
wafer. Because the transmission wavelength can be controlled by simply adjusting the thicknessof the intermediate layer, the fabrication of a mosaic-like filter is practically achievable. First, thebottom-half and intermediate stacks were deposited over the entire area. The deposition
thickness of the intermediate layer was set on the basis of the thickness of the longest-wavelength
bandpass filter [Fig. 7(a)]. The sites to be thinned were then patterned by lithography, followedby etching to achieve the required thickness of the intermediate layer. The same process was
repeated to achieve three different thicknesses [Fig. 7(b)]. The deposition thicknesses were
controlled by adjusting the deposition time, considering the pre-confirmed etching rate. The tophalf of the layers were then deposited over the surface [Fig. 7(c)]. Finally, the filter materials in
the pad areas were etched off for wire bonding [Fig. 7(d)].

(a)

(b)

Fig. 6. (Color online) (a) Multilayer interference filter designs and (b) simulated transmission spectra.
Note that the layer number and the thicknesses are not accurately illustrated in Fig. 6(a).

Sensors and Materials, Vol. 35, No. 9 (2023)

3219

(a)

[STEP 1] Deposit the bottom-half and intermediate layers.

(b)[STEP 2] Pattern the intermediate layer by etching.

(c)[STEP 3] Deposit the top-half layers.

(d)[STEP 4] Open PADs.

Fig. 7.

(Color online) Filter fabrication process.

3.

Results and Discussion

3.1

Spectral sensitivity and imaging results of the fabricated image sensor

The spectral responses of the IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm) pixels of thefabricated sensor are shown in Fig. 8. With illumination perpendicular to the sensor surface, thespectral responses in the NIR region were measured every 10 nm using a monochromator. The
spectral response [Fig. 8(b)] indicates that the three wavelengths of IR1 (780 nm), IR2 (850 nm),and IR3 (920 nm) were captured and isolated by the three different NIR multispectral filters.
Two different IR1 (780 nm) filters indicated similar spectral responses despite the different
adjacent filters. IR2 (850 nm) exhibited the highest peak, followed by IR1 (780 nm) and IR3 (920nm).
The spectral responses are the products of the spectral sensitivity of the base sensor and filtertransmittance. To evaluate the performance of the filters, the actual transmittance of thefabricated sensor was determined by dividing the spectral response by the base sensor sensitivity.
These actual transmittances were then compared with the designed transmittance spectra of themultilayer interference filters, as shown in Fig. 9. The simulated peak transmittances of IR1,
IR2, and IR3 were all almost 100%, and the average transmittances in the wavelength range of±35 nm centered at each peak wavelength were 60, 60, and 65%, respectively. Here, the averagevalues were calculated by dividing the integrated spectral transmittance over the wavelength by
the integration range. Moreover, the ratios between the actual and designed peak transmittancesof IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm) were 61, 85, and 72%, respectively. The actual
transmittance was the lowest in the IR1 (780 nm) filter, as analyzed by the 3D simulation [Figs.
4(c)–4(e)]. However, it was consistently observed that the actual transmittance of the fabricatedfilters was lower than the results obtained from 3D simulations; the actual transmittances were61% compared with 64% obtained from the 3D simulation for IR1 (gap 3%), 85% compared with

3220

Sensors and Materials, Vol. 35, No. 9 (2023)(a)Fig. 8.

(b)(Color online) (a) Fabricated sensor. (b) Spectral response of each multispectral filter in NIR region.

(a)

(b)

(c)

Fig. 9. (Color online) Actual and designed transmittances: (a) IR1 (780 nm), (b) IR2 (850 nm), and (c) IR3(920 nm).

96% for IR2 (gap 11%), and 72% compared with 96% for IR3 (gap 24%). It is considered that thecross-talk in the fabricated sensor not only occurs in the multilayer filter layers but also in thesilicon layer. In contrast, the 3D simulation accounts for cross-talk only in the multilayer filter
layers. It was also observed that the gap between the actual results and the 3D simulations waslarger at longer wavelengths. It is considered that a longer wavelength penetrates deeper into thesilicon and crosses the pixels more. Cross-talk prevention is one of the key opportunities for
improving the transmittance and spectral performance of multispectral imaging sensors infuture research.
The fabricated NIR multispectral imaging sensor was installed in an NIR color camera. The
NIR color technique is to generate RGB color estimates from NIR multispectral signals.(18–21)
Figure 10 shows the outcome of processing the RAW images using the previously describedmethodology(11) for improving color quality. The pixel values obtained from the fabricatedimaging sensor with the described spectral response were converted to values in ideal sensitivity
characteristics modeled as normal distributions with a standard deviation of 15 nm centered at

Sensors and Materials, Vol. 35, No. 9 (2023)

3221

Fig. 10. (Color online) NIR color image obtained using the fabricated imaging sensor.

each peak wavelength. The conversion matrix M shown in Eq. (1) was used for the color spaceconversion. M was applied as in Eq. (2). Here, the pixel values of each channel (IR1, IR2, andIR3) acquired by the imaging sensor are indicated by the subscript ‘raw’, and the ones afterconversion are indicated by the subscript ‘conv’.
 2.668 −0.564 −1.103
 −0.749 2.477 −0.728
M= −0.381 −0.715 2.097 (1)

 IR1conv  IR1raw  IR 2
 = M  IR 2 
conv 
raw  IR3conv  IR3raw (2)After the color space conversion process, the values of IR1conv, IR2conv, and IR3conv were
subsequently reassigned to the R, B, and G values, respectively, in the sRGB color space,followed by brightness adjustment. Because the imaging sensor captures three different
wavelengths, IR1 (780 nm), IR2 (850 nm), and IR3 (920 nm), an NIR color image was clearlydelivered. The image is capable of differentiating the colors of objects as the colors of the ink
bins, plant, and condiment bottles.
3.2

Fundus images and evaluation

The fabricated NIR multispectral imaging sensor was installed in a fundus camera system[Fig. 11(a)]. The sensor was mounted on a sensor board and connected to an image-processing
board. While trans-palpebral, trans-scleral, and trans-pars-planar illumination methods fromoutside the pupil have been proposed for NIR illumination,(22–24) the optical system for thisstudy was built using a Maxwellian illumination system such that collimated NIR light isreflected by a beam splitter and focused on the surface of the eye because it is widely used inconventional fundus cameras and less risky to accidental eye damage. Light is then spread to the3222

Sensors and Materials, Vol. 35, No. 9 (2023)

(a)

(b)

Fig. 11. (Color online) Glare-free NIR color fundus camera with the fabricated sensor. (a) Camera components and(b) an appearance image.

(a)

(b)

Fig. 12. (Color online) Fundus images taken by glare-free NIR color fundus camera: (a) model and (b) human eyes.

retina by the ophthalmic lens. The light source used in the experiment was composed of LEDs
assembled in a fiber line and emitted three wavelengths of IR1 (780 nm), IR2 (850 nm), and IR3(920 nm).
To detect lifestyle-related diseases, fundus imaging was performed by focusing on the bloodvessels around the optic disc. The shutter was opened when the illumination flashedspontaneously. The amount of illumination was controlled according to eye safety guidelines.
Figure 12 shows the acquired fundus images of model and human eyes. The optic disc andblood vessels were clearly visualized. The images were reviewed and confirmed by a medical
doctor that the relative diameter and shape of blood vessels were sufficiently recognizable toobserve the attenuation of retinal arterioles, focal arteriolar narrowing, the broadening of thearteriolar reflex, and minimal arteriolar-venous crossing defects. Consequently, hypertensive
and atherosclerotic lifestyle-related diseases can be diagnosed on the basis of Scheie
classification.(25) Thus, it is confirmed that an NIR glare-free fundus camera with a 4-μm-pixel
pitch NIR multispectral imaging sensor can provide medically relevant and meaningful
information.

Sensors and Materials, Vol. 35, No. 9 (2023)

4.

3223

Conclusions

A 4-μm-pixel pitch NIR multispectral imaging sensor was fabricated by directly depositing
NIR multispectral filters on a sensor wafer. The sensor was applied to a glare-free NIR color
fundus camera, and it was confirmed that the resulting camera was capable of providing
medically relevant and meaningful information. As the goal of developing a “fundus camera
anyone can use” is to make it compact and affordable, the successful outcomes of the fabricatedsensor represent a significant step towards achieving this objective for healthcare purposes. Oneof the primary challenges that need to be addressed for the future improvement of NIR
multispectral imaging sensors is the issue of cross-talk between pixels.

Acknowledgments
This study was supported by the Japan Science and Technology Agency, ACCEL(JPMJAC1601).

References
1 A. Kifley, J. J. Wang, S. Cugati, T. Y. Wong, and P. Mitchell: Microcirculation 15 (2008) 373. http://doi.wiley.
com/10.1080/10739680701812220
2 R. Poplin, A. V. Varadarajan, K. Blumer, Y. Liu, M. V. McConnell, G. S. Corrado, L. Peng, and D. R. Webster:Nat. Biomed. Eng. 2 (2018) 158. https://www.nature.com/articles/s41551-018-0195-0
3 B. C. Bucca, D. M. Maahs, J. K. Snell-Bergeon, J. Hokanson, S. Rinella, F. Bishop, A. Boufard, J. Homann,
C. Y. Cheung, and T. Y. Wong: J. Diabetes Complicat. 32 (2018) 234. https://linkinghub.elsevier.com/retrieve/
pii/S1056872716308145
4 X. Hadoux, F. Hui, J. K. H. Lim, C. L. Masters, A. Pébay, S. Chevalier, J. Ha, S. Loi, C. J. Fowler, C. Rowe,
V. L. Villemagne, E. N. Taylor, C. Fluke, J.-P. Soucy, F. Lesage, J.-P. Sylvestre, P. Rosa-Neto, S.Mathotaarachchi,
S. Gauthier, Z. S. Nasreddine, J. D. Arbour, M.-A. Rhéaume, S. Beaulieu, M. Dirani, C. T. O. Nguyen,
B. V. Bui, R. Williamson, J. G. Crowston, and P. van Wijngaarden: Nat. Commun. 10 (2019) 4227. https://doi.
org/10.1038/s41467-019-12242-1
5 T. T. Nguyen, R. Kawasaki, J. J. Wang, A. J. Kreis, J. Shaw, W. Vilser, and T. Y. Wong: Diabetes Care 32 (2009)
2075. https://doi.org/10.2337/dc09-0075
6 M. Sobue, H. Takata, H. Takehara, M. Haruta, H. Tashiro, K. Sasagawa, R. Kawasaki, and J. Ohta: J. Robot.
Mechatron. 34 (2022) 1152. https://www.fujipress.jp/jrm/rb/robot003400051152
7 Z. Wang, M. Uemura, H. Takehara, M. Haruta, H. Tashiro, K. Sasagawa, and J. Ohta: Jpn. J. Appl. Phys. 60(2021) SBBL07. https://iopscience.iop.org/article/10.35848/1347-4065/abea4c
8 A. Pujari, G. Saluja, D. Agarwal, A. Sinha, A. P R, A. Kumar, and N. Sharma: Curr. Eye Res. 46 (2021) 1605.
https://www.tandfonline.com/doi/full/10.1080/02713683.2021.1958347
9 C. H. Tan, B. M. Kyaw, H. Smith, C. S. Tan, and L. T. Car: J. Med. Internet Res. 22 (2020) e16658. http://www.
jmir.org/2020/5/e16658/
10 M. W. M. Wintergerst, L. G. Jansen, F. G. Holz, and R. P. Finger: Asia-Pac. J. Ophthalmol. 9 (2020) 308.
https://journals.lww.com/10.1097/APO.0000000000000303
11 H. Takehara, Z. Wang, H. Tang, N. Kishida, Y. Horiki, M. Sobue, M. Haruta, H. Tashiro, K. Sasagawa, and
J. Ohta: ITE Trans. Media Technol. Appl. 10 (2022) 59. https://www.jstage.jst.go.jp/article/mta/10/2/10_59/_
article12 C. F. Han, J. M. Chiou, and J. F. Lin: Sensors 20 (2020) 3062. https://www.mdpi.com/1424-8220/20/11/3062
13 S. Yokogawa, I. Oshiyama, H. Ikeda, Y. Ebiko, T. Hirano, S. Saito, T. Oinoue, Y. Hagimoto, and H. Iwamoto:
Sci. Rep. 7 (2017) 3832. https://www.nature.com/articles/s41598-017-04200-y
14 M. A. Green and M. J. Keevers: Prog. Photovoltaics Res. Appl. 3 (1995) 189. https://onlinelibrary.wiley.com/
doi/10.1002/pip.4670030303
15 H. Takehara, H. Sumi, W. Ze, T. Kondo, M. Haruta, K. Sasagawa, and J. Ohta: Proc. 2019 IEEE Biomedical
Circuits and Systems Conference (IEEE, 2019) 1–4.

3224

Sensors and Materials, Vol. 35, No. 9 (2023)

16 Bayer. B: US Patent 3971065 (1976).
17 H. A. Macleod: Thin-Film Optical Filters (CRC Press, Taylor & Francis Group, FL, 2010) 4th ed., Chap. 2.
18 M. Sobue, H. Otake, H. Takehara, M. Haruta, H. Tashiro, K. Sasagawa, and J. Ohta: SPIE Int. Soc. Opt. Eng. 61(2022) 063107. https://doi.org/10.1117/1.OE.61.6.063107
19 H. Sumi, H. Takehara, S. Miyazaki, D. Shirahige, K. Sasagawa, T. Tokuda, Y. Watanabe, N. Kishi, J. Ohta, and
M. Ishikawa: Proc. 2018 IEEE Symposium on VLSI Technology (IEEE, 2018) 163–164.
20 Yasushi Nagamune: US Patent 8854472 (2014).
21 Yasushi Nagamune: US Patent 8836795 (2014).
22 D. Toslak, D. Thapa, Y. Chen, M. K. Erol, R. V. P. Chan, and X. Yao: Opt. Lett. 41 (2016) 2688. https://opg.
optica.org/abstract.cfm?URI=ol-41-12-2688
23 B. Wang, D. Toslak, M. N. Alam, R. V. P. Chan, and X. Yao: Sci. Rep. 8 (2018) 8768. https://www.nature.com/
articles/s41598-018-27112-x
24 C. L. Shields: Arch. Ophthalmol. 121 (2003) 1603. http://archopht.jamanetwork.com/article.aspx?doi=10.1001/
archopht.121.11.1603
25 R. E. Rakel and D. Rakel: Textbook of Family Medicine (Elsevier, PA, 2011) 8th ed., Chap. 41.

About the Authors
Motoshi Sobue received his MS degree in engineering from Waseda
University, Japan, in 1989 and his MA degree in economics from Duke
University, USA, in 1996. He worked for the Bank of Japan, Intel, Dell, andBAT. In 2021, he joined the Graduate School of Material Science, Nara
Institute of Science and Technology (NAIST), Nara, Japan, as a Ph.D.
candidate. He is currently the CEO of Nanolux Co. Ltd.
Hironari Takehara received his ME degree in applied chemistry from Kansai
University, Osaka, Japan, in 1986, and his Ph.D. degree in materials sciencefrom Nara Institute of Science and Technology (NAIST), Nara, Japan, in 2015.
From 1986 to 2012, he was a semiconductor process engineer at Panasonic
Corporation, Kyoto, Japan. In 2015 and 2019, he joined NAIST as a
postdoctoral fellow and an assistant professor, respectively. His current
research interests involve CMOS image sensors and bioimaging.
Makito Haruta received his MS degree in biological science and Dr. Eng.
degree in material science from Nara Institute of Science and Technology(NAIST), Nara, Japan, in 2011 and 2014, respectively. He joined NAIST in2016 as an assistant professor. In 2019, he joined the Graduate School of
Science and Technology of NAIST as an assistant professor. His researchinterests include brain imaging devices for understanding brain functions
related to animal behaviors.

Sensors and Materials, Vol. 35, No. 9 (2023)

3225

Hiroyuki Tashiro received his ME degree from Toyohashi University ofTechnology in 1996 and his Ph.D. degree from Nara Institute of Science andTechnology (NAIST) in 2017. In 1998, he joined Nidek Co., Ltd., working on
the R&D of ophthalmic surgical systems and retinal prostheses. He has been
an assistant professor in Kyushu University since 2014 and an associate
professor at NAIST since 2019. His current research interests include artificial
vision systems and neural interface.
Kiyotaka Sasagawa received his BS degree from Kyoto University in 1999
and his ME and Ph.D. degrees in materials science from NAIST, Japan, in
2001 and 2004, respectively. Then, he worked as a researcher at the National
Institute of Information and Communications Technology, Tokyo. In 2008, hejoined NAIST as an assistant professor and was promoted to an associate
professor in 2019. His research interests include bioimaging, biosensing, andelectromagnetic field imaging.
Ryo Kawasaki received his MD (1997) and Ph.D. (2008) degrees from
Yamagata University, Japan, and MPH degree (2007) from Johns Hopkins
University Bloomberg School of Public Health, USA. He is a physicianscientist in ophthalmology. His current research interests include retinal image
analysis using artificial intelligence, epidemiology, preventive medicine, andcommunity health.
Jun Ohta received his ME and Dr. Eng degrees in applied physics from The
University of Tokyo, Japan, in 1983 and 1992, respectively. In 1983, he joined
Mitsubishi Electric Corporation, Japan. In 1998, he joined Nara Institute of
Science and Technology (NAIST), Japan, and was appointed as a professor in
2004. His research interests include smart CMOS image sensors forbiomedical applications. He is a Fellow of IEEE, the Japan Society of Applied
Physics, and ITE.