Patent Document ID: 20160117800
Application ID: 14981518
Patent Status: 0

Claim One:
1. A photographic image acquisition method for increasing resolution of acquired images comprising: providing a primary image sensor optically coupled to a primary imaging lens having a primary optical axis, providing at least one secondary image sensor optically coupled to a secondary imaging lens having a secondary optical axis, wherein the primary and secondary optical axes are set substantially parallel to each other, and wherein the primary and secondary image sensors are set in substantially the same geometric plane such that their focal planes arrays receive optical projections of substantially the same scene, and wherein the primary and secondary image sensors comprise substantially identical focal plane arrays of substantially identical photosensitive elements, and wherein the primary sensor is a multi-spectral image sensor comprising a Bayer color filter array deposited on top of its focal plane array deposited on top of its focal plane array, and the secondary image sensor is a panchromatic sensor, providing computational means for computationally reconstructing the super-resolved multi-spectral images from the images acquired by the primary and the secondary image sensors, wherein said computational reconstruction of the super-resolved multi-spectral image advantageously exploits nonlocal self-similarity and redundancy at the scale of small image patches between the multi-spectral and the panchromatic images, thereby advantageously admitting mismatches between the two images due to parallax, performing an initial estimate of the intended super-resolved multi-spectral image extracting a luminosity subcomponent from the multi-spectral image, and upsampling said luminosity subcomponent image to bring it up to the intended scale of the super-resolved image, wherein luminosity pixel values are extracted from the multi-spectral image by using a mixture of spectral subcomponent pixel values at predetermined ratios, for each pixel of said upsampled luminosity image, extracting a small image patch surrounding said pixel and extracting a plurality of smaller image patches from the panchromatic image, such that the size of said interpolated luminosity patch and the size of each panchromatic patch are related by the same ratio as the respective sizes of the two images, upsampling each patch of the plurality of extracted panchromatic patches to the same scale as the aforementioned upsampled luminosity patch, or conversely downsampling said luminosity patch to the same scale as the plurality of extracted panchromatic patches, matching each patch of said plurality of panchromatic patches against said luminosity patch brought to the same scale as said plurality of panchromatic patches using a measure of patch similarity, calculating a new pixel value from multiple pixel values at the centers of said panchromatic image patches by weighing their relative contributions of said multiple pixel values in proportion to their similarity to said luminosity patch, inserting said new pixel value into the upsampled luminosity image at the coordinates corresponding to the center of said luminosity patch thereby replacing the original pixel value at these coordinates, iterating over all pixels of said upsampled luminosity image to gradually replace all of its pixels to form a new higher-resolution image as an improved estimate of the intended super-resolved image in place of the initial estimate, repeating this iterating step one or more times over the entire image, using said previous iteration image as the new estimate for the next iteration, each iteration thereby forming a new higher-resolution image as an improved estimate of the intended super-resolved image in place of the image formed at the previous iteration, and wherein one or more of said iterations additionally involves regularization using Total Variation regularization or Tikhonov regularization, wherein one or more of said iterations additionally involves a reversal of optical blur applied to the newly obtained estimate of the intended super-resolved image, and wherein said plurality of panchromatic patches may be extracted not only from a given panchromatic image, but also from multiple temporally proximal panchromatic images (video frames) acquired shortly before and after the given image, upsampling a chromaticity subcomponent from the multi-spectral image to the scale of the super-resolved luminosity image, and adjusting the chromaticity subcomponent pixel values of the multi-spectral image to avoid any color drift as a result of the substitution of the original luminosity pixel values with the generally different panchromatic values.