公益財団法人 セコム科学技術振興財団
研究成果報告書

研究課題名
次世代画像入力システムを実現する高速パンチルト・リフォーカスカメラの研究
Developments of High-Speed Pan-Tilt and Refocus Camera
for Next-Generation Image Capture System

研究期間
平成２８年１０月 ～ 令和３年５月

報告年月平成３年６月
研究代表者
東京農工大学大学院 工学研究院学部
高木 康博

教授

Insitute of Engineering, Tokyo University of Agriculture and Technology
Yasuhiro Takaki

1

Abstract
Cameras are important to ensure the safety of nowadays life of people. The recent advances of the image
recognition using artificial intelligence has also increased the importance of cameras. Recently, theperformances of cameras have been improved, such as resolution and frame rate. However, the movement of
the gaze direction of cameras (pan-tilt function) has so far been performed using the mechanical system so thecameras cannot follow fast movements of target objects. The focus of the camera has also been altered using
the mechanical system so the cameras cannot quickly change the focus position. This study proposes thehigh-speed pan-tilt refocus camera to address the remaining issues of the present cameras. The experimental
verification of the proposed technique is shown and the applications of the proposed technique are discussed.
The high-speed pan-tilt refocus camera consists of the pan-tilt part consisted of micro-lens arrays
and the refocus part which performs numerical image focusing using a computer (refocus process). In thepan-tilt part, two parallelly aligned micro-lens arrays are used, which are shifted each other to change thedirection of the optical axis so the pan-tilt function is realized. When the lens arrays are shifted with anamount of the pitch of lenses of the lens arrays, the maximum pan-tilt angle is achieved. Therefore, when thelens arrays with a lens pitch of approximately 100 µm are used and piezo actuators are used to displace thelens arrays, the high-speed pan-tilt action is achieved. The aperture array is used to extract partial images fromimages produced by the first lens array and the extracted images are magnified by the second lens array toproduce elementary images on the image sensor. The parallax images are synthesized from the elementary
images and the numerical focusing is performed by synthesizing the parallax images. Because the parallelismof this refocusing calculation is quite high, this study utilizes GPUs to enable the fast numerical focusing.

The experimental system was constructed to verify the proposed technique. The micro-lensarrays with a lens pitch of 103.5 µm and a size of 15.0 mm×11.0 mm were designed and fabricated.
The image sensor with a resolution of 4,096×3,000 and the piezo actuators with a maximum strokeof 100 mm were combined to build the experimental system. The experimental system had amaximum pan-tilt angle of 25⁰ and an operating frequency of 1 kHz. The multiple GPU system wasconstructed for the refocusing process and the calculation time of 3.8 ms was achieved using twoGPUs. The above results show the validation of the proposed technique.
Then, the reduction of blurs of the refocused images was explored. When the conventional
technique is used, the parallax images are distorted due to the lens aberration of the lens arrays andthe synthesis of the distorted parallax images causes blurs in the refocus images. This study proposesa new algorithm which generate refocus images directly from the elementary images without thegeneration of the parallax images. This algorithm reduced the blurs of the refocus images. Because
the parallax images are not generated, the calculation time and the memory usage are reduced.
The applications of the proposed high-speed pan-tilt refocus camera were developed, suchas, the resolution increased image capturing technique by use of the pan-tilt function, and the 3D
image capturing system for 3D displays by use of the refocusing function.
Finally, the future improvements of the performances of the high-speed pan-tilt refocus
camera and its applications are discussed.
3