IMAGE PROCESSING APPARATUS

Abstract: In order to correct the swathe related banding effect often seen on swathe-scanned imagers a vertical smoothing function separates out artifact detail from the idealised image and derives from this, proportional corrections for each channel within a swathe. In cases where the artifact varies from swathe to swathe a second algorithm is used to obtain correction values for each line of the image.

Description: This invention relates to an image processing apparatus, and in particular it relates to apparatus of the type which includes a number of channels of detectors and scans an image in swathes, each swathe comprising a simultaneous scan of all of the channels. Such systems are commonly used in infra-red and other radiation imaging systems. However, since there are inevitably variations in detector element responsivities and offsets, a degree of signal processing is generally required to match the detector signals. Such a signal processing technique usually makes one or more assumptions, the most common of which is that the portions of the image viewed by each detector element during one entire field or frame scan are essentially the same, i.e., the average image intensity viewed by the first detector element over the entire frame or field is equal to the average intensity viewed by the n'th detector. One example of a signal processing system operating in this way is disclosed in GB-A-1588018 which alters the gain applied to the output of each detector element based on the assumption that the average intensity of the image seen by each detector element over a period of time is identical. For the majority of images this assumption is reasonable and good matching between channels is achieved. However, for certain images these assumptions are not valid, particularly where there is a temperature gradient across a scene such as a transition between sky and ground. This results in a swathe related banding structure becoming evident in the displayed image in which bands are seen on the image of width equal to the swathe width and the swathe boundaries are highly noticeable. The degree of degradation of the image will of course depend upon the actual scene being imaged. It is convenient to think of these degraded images as the sum of the ideal image and a so called "artifact" image which varies in the vertical image direction but is substantially constant in the horizontal direction, the artifact image being that portion of the image which differs from the idealised image. Although the artifact image may show substantial differences from line to line within a swathe, due to differences in the gains and offsets of individual detectors, the artifact image of an individual channel often only varies slowly in the vertical image direction. This results in the artifact portion of the image comprising a slowly varying structure that is repeated for each swathe in the image. Thus a banded structure is seen. The present invention arose from a need to provide a method for removing the artifact and producing a clearer and better quality final image. According to the present invention there is provided apparatus for processing a swathe-scanned input image, which image is of the type comprising a plurality of lines which together form a representation of the image being scanned, respective groups of lines being scanned together in swathes, the input image comprising an ideal image component and an artifact image component, the artifact image component representing deviations of the input image from the ideal image component, the artifact image component comprising a swathe-related component; characterised by means for deriving a first set of values, each value being representative of the average intensity of an image line; means for removing low frequency image structure from the first set of values to produce a second set of values, said second set of values being substantially representative of the artifact image component; means for smoothing the second set of values to produce a set of correction values for applying to the input image, and; means for applying the correction values to the input image so as to produce an output image having a reduced artifact image component. By adjusting the outputs in this way a type of smoothing which will be referred to as vertical smoothing is applied to the image and the degrading "banding" artifact, seen when the image is a swathe scanned one, is noticeably reduced. In one embodiment of the invention the means for deriving the average intensity of each respective line comprises an accumulator to which the intensity values of each pixel within the line are applied. In a preferred embodiment each respective one of the second set of values is the difference between each respective one of the first set of values and a local average value of the average intensity over a chosen number of adjacent lines. The apparatus may include means for calculating the local average over a number of values equal to the number of lines of each swathe. A single correction value may be derived for each channel within the swathe. This correction value may be equal to the mean or medium of the second set of values for that channel, over the entire image. In order to overcome the problem of anomalous errors being introduced into the correction signal by derived error values arising from objects of noticeably different intensity within the image, the apparatus may further include a filtering function adapted to act on a chosen number of adjacent error values for each channel in turn and to obtain a median value for those error values which is used to obtain the average error vale for that channel over the entire image. The filtering function may be a 3 point median filter adapted to pass over each channel in turn. The correction function may be applied to the channel outputs through a feed forward or a feed back arrangement. Embodiments of the invention will now be described by way of example only with reference to the accompanying drawings in which; Figure 1 shows an application of the present invention in a feed forward mode. Figure 2 shows an application of the present invention in a feed back mode. Figure 3 shows schematically how the artifact image exhibits its effects. Figure 4 shows the affects of smoothing and error calculation on an image with a notable object. Figure 5 shows how the application of a filter can increase the performance of the system, and; Figure 6 shows a schematic digital correction circuit. Figure 1 shows only those parts of an imaging system which are relevant to the present invention. A typical imager comprises a number q of channels of detectors, which are scanned simultaneously in swathes across a scene in the normal manner. As described above, without further correction processing the normal processing of such signals may give a banded swathe related structure. Such an image, after normal processing and hence already containing the artifact detail, and therefore exhibiting banding, is applied as the input to the apparatus shown in Figure 1, which therefore does not actually cure the cause of the artifact image but rather applies post processing to subsequently remove the artifact image before display of the resulting image. The system comprises three processing units, which perform respectively a measurement function 1, a processing function 2 and a correction function 3. After correction, the video signals are output for display or further processing as may be required. Figure 2 shows an alternative arrangement in which the three functional boxes corresponding to those in Figure 1 are used in a feed-back rather than a feed-forward system. The embodiment of Figure 2 will not be further described in this specification but will be easily derivable by one skilled in the art. The sequential input from all the detector channels is firstly acted upon by the measurement function 1. This is designed to produce a minimum set of image parameters which are to be used by the processing function to determine the exact form of the artifact image. Generally, the artifact image will be substantially constant in the horizontal image direction, i.e. the swathe direction, and hence it is found that a suitable parameter is the mean intensity of each image line. Typically this may be achieved in a digital system by accumulating the pixel intensity values along each line. If the number of pixel values is chosen to be for example a power of 2 then the output of the accumulator, suitably shifted, will provide an output representing the mean line intensity. An example of the result of the measurement function is shown in Figure 3, which figure is schematic since the average intensity of each line within a swathe is unlikely to be the same. However, the resulting pattern is likely to be repeated from swathe to swathe. Since the image is averaged in the horizontal direction much of the original image structure is suppressed, leaving the artifact image unaffected. Hence, the resulting signals generated by the measurement function comprise the low frequency vertical component of the original image, i.e. without the low amplitude high frequency components expressing the fine detail, together with the artifact image components. However, anomalous errors consisting of large amplitude high frequency image components may still be apparent in the output of unit 1. The measured set of data line values are subsequently processed by processing unit 2 to calculate a set of corrections for each image. This is basically achieved by extracting the artifact image structure from the set of line averages and generating a signal which is in effect an inverse of the artifact image, thus serving to cancel out the effects of the artifact image. Artifact images can generally take one of two forms, either as a constant structure down the image, i.e. appearing like a repeated structure on a diagram similar to Figure 3, or one which changes in magnitude and/or sense down the displayed image. In the following description the image is defined as comprising r swathes, each swathe comprising q lines produced by q detector channels. Hence the total number of image lines, n, is given by: n = r.q An image line may be referred to either by its line number L, where 1 ≦ L ≦ n, or by its swathe number j where 1 ≦ j ≦ r and its channel number i where 1 ≦ i ≦ q. These parameters are related by: L = i+(j-l).q The set of line averages produced by the measurement function 1 may be represented as the set of x L or x ij . The first process that is applied is intended to remove low frequency image structure from the set of line averages xL, thus producing a set of error estimates e L for each line. These will closely approximate to the artifact image component in the image although perhaps may include some high frequency image component detail. Any suitable method may be used to achieve this although one method is to calculate the difference between each measured x L and a local average value for the x L . Reference to Figure 3 will show that an effective local average can be obtained by averaging q adjacent x L values, For the case when there is no variation from swathe to swathe in the structure a first algorithm according to the present invention averages all the derived errors e L (or e ij ) for each channel so that the correction for channel i of any swathe is defined by i.e. the algorithm calculates q corrections for each image, one correction for each channel. This first simple algorithm is effective for most images in which there are no variations in the vertical direction from swathe to swathe. For the case where the banding structure does vary from swathe to swathe a second algorithm is used. This calculates a correction for each image line that is the inverse of the error estimate of that line. However, it will be appreciated that these error estimates can not be used unmodified with some images since some of the values derived will have resulted from particular image structure rather than artifact image detail. These anomalous values must be removed to prevent introducing anomalous errors. A common situation where this applies is shown in Figure 4 where Figure 4a shows a typical swathe related banding effect seen with an image with an object of noticeably different intensity to the background and having sudden changes in the vertical direction. Figure 4b shows the same image after smoothing and Figure 4c shows the effect of applying the algorithm described above to obtain error values e ij . It is seen that at the boundaries of the object the error values derived are large and hence the use of the unmodified error values would tend to introduce anomalous errors into the image, thus degrading it. Accordingly this second algorithm includes a filtering function which filters the error values e L on a channel by channel basis. i.e. in which the calculated correction values for lines corresponding to a particular channel depend on the error estimates for that channel only. Figure 5 shows one example where a three point median filter 4 is used. An 'n' point median filter as used in the embodiment is a filter which replaces each input value with the median of n local input values; so, in the present case each value is replaced by the median of that value and the immediately adjacent values for that channel, as is shown in Figure 5. By using such a filter, anomalous errors arising from image detail such as that shown in the larger error value of channel one in the third swathe of the Figure are removed. Hence, any anomalous errors caused by the inclusion of high frequency image structure that stand out against the generally smoothly changing swathe errors are removed by the median filtering process. The derived correction values are subsequently applied to the correction function unit 3. This then applies the correction values to each respective line of the image to cancel the artifact image. The calculated correction values may be applied by either analogue or digital means by methods well known in the art. An analogue correction method may introduce a proportional dc offset in the image signal for each respective line. A digital processing method may require the corrections to be added to each sample or pixel on each line. A schematic digital system is shown in Figure 6 in which the uncorrected digital image data is applied to a digital adder 5. The correction values for each line are held in a store 6 and added, pixel by pixel, to the uncorrected data in the adder, dependent upon the line and/or channel to which the data pertains, such that corrected data is output on line 7. A clock 8 is used to synchronise the system and the correction values are fed into store 6 during an initial scan. These values can be updated as required. In use, one correction value for the first line may be added to each pixel of that line in turn, a second value to each pixel of the second line and so on.

Claims: Apparatus for processing a swathe-scanned input image, which image is of the type comprising a plurality of lines which together form a representation of the image being scanned, respective groups of lines being scanned together in swathes, the input image comprising an ideal image component and an artifact image component, the artifact image component representing deviations of the input image from the ideal image component, the artifact image component comprising a swathe-related component; characterised by means (1) for deriving a first set of values, each value being representative of the average intensity of an image line; means (2) for removing low frequency image structure from the first set of values to produce a second set of values, said second set of values being substantially representative of the artifact image component; means (2) for smoothing the second set of values to produce a set of correction values for applying to the input image, and; means (5) for applying the correction values to the input image so as to produce an output image (7) having a reduced artifact image component. Apparatus as claimed in Claim 1 wherein the means for deriving the average intensity of each respective line comprises an accumulator (1) to which the intensity values of each pixel within the line are applied. Apparatus as claimed in any of the preceding claims wherein each respective one of the second set of values is the difference between each respective one of the first set of values and a local average value of the average intensity over a chosen number of adjacent lines. Apparatus as claimed in Claim 3, wherein the means (2) for producing the second set of values calculates the local average value over a number of values equal to the number of channels (q) within a swathe. Apparatus as claimed in Claim 3 or 4 wherein the means (2) for deriving the set of correction values is arranged to derive a single correction value for each channel within the swathe, and the single correction value for each channel is derived from the mean or median of the values of the second set of values corresponding to that channel. Apparatus as claimed in any of Claims 1 to 4 wherein the means for deriving the set of correction values comprises means for filtering the second set of values (2) to obtain a unique correction value for each individual line. Apparatus as claimed in Claim 6 wherein the filtering means (2) is arranged to perform a filtering function on the second set of values for each channel individually. Apparatus as claimed in Claim 6 or Claim 7 wherein the filtering means is a three point median filter. Apparatus as claimed in any of the preceding claims, configured as a feed-forward arrangement.

Inventors: DIMMICK, STEPHEN, SAMUELS, MARK ANDREW
Applicants: GEC-MARCONI LIMITED
Application year: 1988
Main classification: H04N5
IPC codes: H04N5