Press Release
PR2020160

Socionext and Osaka University Develop New Deep Learning
Method for Object Detection in Low-Light Conditions
Allows recognition of RAW image data without generating additional dataset
Yokohama, August 21, 2020 --- Socionext Inc. and a research group at Osaka University
Institute for Datability Science, led by Professor Hajime Nagahara, have jointly developed a new
method of deep learning, which enables image recognition and object detection in extremely lowlight conditions. By merging multiple models, the new method enables the detection of objects
without the generation of huge datasets, a task previously thought to be essential.
Socionext plans to incorporate this new method into the company’s image signal processors todevelop new SoCs, as well as new camera systems around such SoCs, for applications including
automotive, security, industrial and others that require high performance image recognition. The
research work will be presented at European Conference on Computer Vision (ECCV) 2020, held
online from August 23 through 28 (British Summer Time). https://eccv2020.eu/
A major challenge throughout the evolution of computer vision technology, has been to improve
the image recognition performance for applications such as in-vehicle cameras and surveillancesystems under poor lighting conditions. Previously, a deep learning method using RAW image datafrom sensors has been developed, called “Learning to See in the Dark” [1]. However, this method
requires a dataset of more than 200,000 images with more than 1.5 million annotations [2] for endto-end learning. Preparing such a large dataset with RAW images is both costly and timeprohibitive.

Fig.1 Learning to See in the Dark / Challenges for RAW image recognition (View Larger Image)

The joint research team proposed the new domain adaptation method, which builds a required
model using existing datasets by utilizing machine learning techniques such as Transfer Learning
and Knowledge Distillation. The new method resolves the challenge through the following steps:

For Press Inquiry:
Socionext Inc.
Tel: +81-45-568-1006

https://www.socionext.com/en/contact/

1 /3

PR2020160(1) building an inference model with existing datasets, (2) extracting knowledge from theaforementioned inference model, (3) merging the models by glue layers, and (4) building
generative model by knowledge distillation. It enables the learning of a desired image recognition
model using the existing datasets (Fig.2).

Fig.2 Domain Adaptation Method (View Larger Image)

Using this domain adaptation method, the team has built an object detection model "YOLO in theDark" using RAW images taken in extreme dark conditions, with the YOLO model [3] (Fig. 3).
Learning of the object detection model with RAW images can be achieved with the existing dataset,without generating additional datasets. In contrast to the case where the object cannot bedetected by brightness enhancement of images with existing YOLO model (a), the proposed newmodel made it possible to recognize RAW images and detect objects (b). The amount of computing
resources needed in this new model is about the half of the baseline model, which uses thecombination of previous models (c).

Fig.3 YOLO in the Dark (View Larger Image)2 /3

PR2020160

This "direct recognition of RAW images" by the method is expected to be used for object detection
in extremely dark conditions, along with many other applications. Socionext plans to incorporate
this new method into the company’s image signal processors to develop new SoCs, as well as newcamera systems around such SoCs, and offer leading edge solutions for applications including
automotive, security, industrial and others that require high performance image recognition.

[European Conference on Computer Vision (ECCV) 2020]
Held online from August 23 to 28 (British Summer Time)https://eccv2020.eu/
Presentation by Yukihiro Sasagawa, Socionext and Hajime Nagahara, Osaka University"YOLO in the Dark - Domain Adaptation Method for Merging Multiple Models -"Notes:[1] “Learning to See in the Dark” : CVPR2018, Chen et al.
[2] MS COCO dataset as an example (https://cocodataset.org/)[3] YOLO (You Only Look Once): One of the deep learning object detection methods

About Socionext
Socionext is a global, innovative enterprise that designs, develops and delivers System-on-Chip solutions tocustomers worldwide. The company is focused on technologies that drive today’s leading-edge applications inconsumer, automotive and industrial markets. Socionext combines world-class expertise, experience, and anextensive IP portfolio to provide exceptional solutions and ensure a better quality of experience for customers.
Founded in 2015, Socionext Inc. is headquartered in Yokohama, and has offices in Japan, Asia, United Statesand Europe to lead its product development and sales activities. For more information, visit
https://www.socionext.com.
All company or product names mentioned herein are trademarks or registered trademarks of their respective
owners. Information provided in this press release is accurate at time of publication and is subject to change
without advance notice.

3 /3