Patent ID: 11922609
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G

Claim 5:
6. A machine vision system, comprising a differentiable image signal processor (ISP) including a block-wise differentiable image signal processor comprising a plurality of functional modules, each functional module of the plurality of functional modules comprising a respective subset of the learned parameters of the differentiable ISP, and each functional module being configured to perform a respective image processing task, each image processing task being selected from a list consisting of black level correction, demosaicing, white balancing, sharpening, color correction, and tone map correction,
wherein the differentiable ISP is trained by:
obtaining a plurality of raw images captured by a sensor, the raw images being of a sensor-specific type;
obtaining a plurality of training images that have been used together with supervised learning to train a perception module to perform an image perception task, the training images being of a perception module-specific type;
training the differentiable ISP, using semi-supervised learning, to process the raw images of the sensor-specific type to generate adapted images of the perception module-specific type, using the plurality of training images and the plurality of raw images, and

wherein the plurality of functional modules comprise, in series:
a black level correction module for processing the raw images to generate black-level corrected images based on learned parameters comprising a black value and a white value;
a demosaicing module for processing the black-level corrected images to generate demosaiced images by performing color-channel filtering based on learned parameters comprising:
three low-pass filters corresponding to three color channels of the black-level corrected images, each low-pass filter comprising four scalar learned parameter values; and
three scalar weight factors for controlling inter-channel correlation of high-frequency characteristics;

a white balancing module for generating white-balanced images by applying gain to each of the three color channels of the demosaiced images based on learned parameters comprising three gain values;
a sharpening module for generating sharpened images by deblurring each color channel of the white-balanced images in the frequency domain based on learned parameters comprising three deblurring matrices corresponding to the three color channels, each deblurring matrix comprising no more than 100 scalar learned parameters;
a color correction module for generating color-corrected images by applying a color correction matrix to the three color channels of the sharpened images, the color correction matrix being a three-by-three matrix comprising nine scalar color correction learned parameters; and
a tone map correction module for generating the adapted images by applying gamma correction and sigmoid curve remapping the color-corrected images, based on learned parameters comprising:
a low-intensity curve parameter for determining a curve to be applied to low-intensity pixels;
a high-intensity curve parameter for determining a curve to be applied to high-intensity pixels; and
a knee parameter for determining a threshold between low-intensity pixels and high-intensity pixels.