Image processing method and apparatus

Abstract: To improve the quality of previewing technique, that is, the technique for displaying on a monitor, before printing, an image to be printed, the image signal for driving the monitor is generated based on the resolution (e.g., dots/inch) of the printer and/or the number of printable tonal levels or gradations of the printer, in addition to known parameters (e.g., color gamut of the printer).

Description: BACKGROUND OF THE INVENTION Field of the Invention This invention is related to an image processing method and apparatus, and in particular to a method and apparatus for displaying an image on a monitor before printing. Related Art Recently, color image processing systems have come into wide use, so that monochrome printers are being replaced by color printers. As a result, the problem of an image on a monitor display not exactly matching the image output by the color printer, has become important. One of the reasons for this problem is that a color printed by the color printer is not necessarily the same as the (ostensibly) same color displayed on the color monitor. Research to solve this aspect of the problem, by ensuring that the color displayed on the color monitor and the color actually printed by a color printer, match, has recently been widely pursued. To this end, research for an adequate preview function (that is, the function of displaying a probable printing color, on the basis of a prediction of how the desired color will be printed by the color printer) has been carried out. A "masking method" and a "look up table method" are examples of well-known techniques. As a result, it is possible to get a better quality image by color image processing than before. However, even with such processing, it is hard to get a more precise preview image, in a case where the kind of output device (printing device) to be used, is changed. EP-A-0534871 describes an image system which is primarily concerned with the fact that different image reproduction devices have different colour spaces or gamuts making accurate colour reproduction difficult. The problem is addressed by enabling colour management to be applied to image data before it is output to a selective image reproduction device so as to achieve "what you see is what you get" colour matching of the image data with respect to the colour fidelity limitations of the intended image reproduction device. SUMMARY OF THE INVENTION An object of the present invention is to improve the quality of a preview image. Another object of the present invention is to provide a method and apparatus for generating an improved preview image. According to one aspect of the present invention, there is provided an image processing method as set out in claim 1. According to another aspect of the present invention, there is provided an image processing apparatus as set out in claim 8. An embodiment of the present invention is to provide a system that will improve the quality of a preview image. Another object of the present invention is to provide software for a computer that will improve the accuracy of a preview image generated by the computer. The foregoing and still other aspects and advantages of the present invention will become fully apparent from the following description of the preferred embodiments, taken in conjunction with the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram for explaining the inventor's analysis of the problem solved by the present invention. Fig. 2 is a block diagram of a first example included for illustrative purposes and not falling within the scope of the claims. Fig. 3 is a perspective view of the ink jet color printer 420 shown in Fig. 2. Fig. 4 is a block diagram for explaining the color image processing performed by the ink jet color printer 420. Fig. 5 is a block diagram illustrating the operation of the host computer shown in Fig. 2. Fig. 6 is a simplified flow-chart for explaining the operation of the system printer selector 580. Fig. 7 is a schematic block diagram for explaining a changeover of the error filter used in binarizing. Fig. 8 is a schematic diagram of a second example included for illustrative purposes and not falling within the scope of the claims. Fig. 9 is a schematic diagram of an embodiment of the invention. Fig. 10 is a flowchart for explaining the operation of the configuration process. Fig. 11 is a diagram showing a quantization error generated by the binarizing process. Fig. 12 is a schematic diagram of an embodiment of the invention, for explaining a relation between the printed image and the monitor image. Fig. 13 is a flow chart for explaining the operation of a parameter producing device in Fig. 12. DETAILED DESCRIPTION OF THE PRESENT EMBODIMENTS The cause of the above-described shortcoming, as analyzed by the inventor, is considered to be the difference between the image recording methods of ink jet printers and color laser printers. Referring to Fig. 1, the monitor 115 and the color laser printer 130 are multivalue output devices, while the color ink the printer 120 is a binary output device. The monitor 115 displays a preview image 111 processed by a computer 110. The characteristics of these output devices are as follows. (1) Multivalue Output Devices The monitor 115 and the color laser printer 130 produce the color image 131 using 256 tones (density levels) for each color component (red, green, blue; "R.G.B") per pixel. (2) Binary Output Device The color ink jet printer 120, in contrast, produces the color image 121 using only 2 tones for each color component R.G.B. per pixel. To operate the binary output device, a halftoning process, typically as a dither method or an error diffusion method, is used. In the halftoning process, a noise component is added to image data to be displayed. The following is clear from the inventor's analysis: the noise added in the halftoning process influences the visual impression made by the image 121 produced by the color ink jet printer 120. More generally, it may be said that the difference in image resolution between these devices, gives rise to an error in the color shown in a preview image. Fig. 2 is a block diagram of a first example included for illustrative purposes and not falling within the scope of the claims. This example has a preview function for changing over the developing process between (a) using the color laser printer and (b) using the color ink jet printer. Referring to Fig. 2, 410 is a host computer for generating a color image. 413 is a monitor display for displaying a generated color image. 420 is an ink jet color printer for receiving color image data from the host computer 410 and for printing a color image based on the received color image data. 421 is a color print output by the ink jet color printer 420. 430 is a color laser printer for receiving color image data and for printing a color image based on the received color image data. 431 is a color print output by the color laser printer 430. 441 is a picture displayed on the monitor 413, predicting a color image to be printed by the ink jet color printer 420, and 442 is a picture displayed on the monitor 413, predicting a color image to be printed by the color laser printer 430. Fig. 3 is a perspective view of the ink jet color printer 420. Referring to Fig. 3, 201 is the body of the ink jet color printer 420, and 211 is a platen for paper P. 212 is an ink jet printing head for depositing ink on the paper P. 213 is a guide member for guiding the ink jet head 212. 214 is a pipe for conducting ink from a pump 217 to the ink jet head 212. 215 is a front guide controller for guiding the paper P along the platen 211. 216 is a driver for driving the guide member 213 so that the ink jet head 212 moves right and left along the guide member 213. Pump 217 is for conducting ink from cartridges 220 to 223 to the ink jet head 212. 218 is a control panel by means of which an operator enters instructions to control the operation of the ink jet printer 420. 219 is a control board which directly controls the operation of the ink jet printer 420. The control board 219 also converts input R.G.B. image data into Y.M.C.Bk (yellow, magenta, cyan, black) image data to drive the ink jet head 212 by color image processing that will be explained below. 220 to 223 are color ink jet cartridges, respectively containing yellow, magenta, cyan and black ink. Each cartridge has a pipe 220a to 223a, respectively, that connects the cartridge to a pressure pump 216. The control board 219 receives an instruction and color image data from the host computer 410, and controls the above-mentioned elements to print a color image based on the received color image data. Color Image Processing In The Ink Jet Color Printer The color image processing for the printer 120 is explained by reference to Fig. 4. Fig. 4 is a block diagram for explaining the color image processing for ink jet printer 420. 31 is input R.G.B. color image data. The R.G.B. color image data 31 is input into the control board 219 through interface 32, and converted into Y.M.C.Bk color image data by converting process 33. The Y.M.C.Bk color image data is developed into dot image data by the developing process 34. The dot image data is binarized by an artificial half toning process, such as the dither method or the error diffusion method, by the binarizing process 36, and is output as printing data. The processes 32, 33, 34 and 36 are performed by hardware or software (printer driver). Fig. 5 is a block diagram of the operation of the host computer 410 for explaining processing performed in this example. In Fig. 5, the explanatory notes show the kind of elements included in each portion. 510 is a file system, and 521 is a load function for loading data from the file system into a user's memory 530. 522 is a save function for saving data from the user's memory 530 to file system 510. 531 is color image data stored by the memory 530. 541 is a display function plus driver for converting the color image data stored by the memory 530 into data suitable to drive a monitor 115. 542 is a preview function for predicting color image data from the color image data 531 so as to display a color image corresponding to a color image about to be printed. 550 is a video memory that comprises a bit map memory 551. 552 is a display interface for interfacing between the monitor display 115 and the video memory 550. 573 is a print function and driver for converting the color image data 573 into data suitable to be output to the printer 120. 574 is a print buffer memory for storing the image data converted by the print function and driver 573. 575 is a printer interface for interfacing between the print buffer memory 574 and the printer 120. 580 is a system printer for changing over the print function and driver 573. The operation of the structure shown in Fig. 5 will now be explained. a) The color image data to be processed is loaded in user's memory 530 from a film system 510 through the load function 521. b) The color image data 531 stored by the user's memory 530 is edited by an editing function, or is saved in the file system 510 if necessary. c) In order to display a color image on the monitor display 115 in the usual manner the color image data stored by the user's memory 530 is processed by the display function and driver 541 and is written in the bit map memory 551. After that the color image data stored by the bit map memory 551 is displayed on the monitor 115 by the display interface 552. d) In order to display a color image on the monitor display 115 as a preview image, however, the color image data stored by the user's memory 530 is processed by the preview function 542 and written in the bit map memory 551. After that the color image stored by the bit map memory 551 is displayed on the monitor 115. At this time, the preview function 542 is operated by the print function 573. e) In order to print a color image based on the color image data 531 stored by the user's memory 530, the color image data 531 is converted by the print function and driver 573 and is written into print buffer memory 574. The color image data written in the print buffer memory 574 is output to the printer 120 through the printer interface 575. At this time, the system printer selector 580 changes over between the preview function 542 and printer interface 575 in accordance with the characteristics of the selected printer. System Printer Selector Fig. 6 is a simplified flowchart for explaining the operation of the system printer selector 580 shown in Fig. 5. Referring to Fig. 6, file system 510 contains not only the color image data but also a printer driver 631 for the color ink jet printer 120, and a printer driver 632 of the color laser printer 130. The color ink jet printer 120 is connected at CH1 within the printer interface 575. The color laser printer 130 is connected at CH2 within the printer interface 575. The operator of the host computer 110 selects a printer by selecting one of the printers on the Printer List displayed on a window of the monitor 413 of the computer 110 shown in Fig. 2. The Printer List is shown at 670 in Fig. 6. The system printer selector 580 operates as follows. a) To load a printer driver stored in the file system 510 into print function and driver 573 in accordance with a selected printer. b) To change channels in the printer interface 575 in accordance with the selected printer. c) To transmit a signal to the preview function 542 so that the preview function recognizes the printer driver stored by the print function and driver 573 to the selected printer. A solid line in Fig. 6 represents a data flow when the color ink jet printer 120 is selected. The printer driver A stored in the file system 510 is loaded into the print function and driver 573, and channel CH1 of the printer interface is selected. The preview function 542 can access the print function and driver 573 based on the signal 625. To make a preview image corresponding to the color image data 531, the preview function 542 calls the print function driver.573, and function 542 makes a binarized image using binarizing process 36 shown in Fig. 4. When the color laser printer 130 is selected, the printer driver B stored in the file system 510 is loaded into the print function and driver 573, and channel CH2 of the printer interface is selected. The preview function 542 can access print function and driver 573 based on the signal 625. To make a preview image corresponding to the color image data 531, the preview function 542 calls the print function and driver 573, and function 542 makes a tone (binary) processed image by a tone process corresponding to a tone process that is used in the color laser printer 130. According to the present example, the preview function 542 selects the print function and driver in accordance with a printer selected between the printer 120, which is a binary output device, and the printer 130, which is a multivalue output device, and therefore, the preview image can be more precise. Second example included for illustrative purposes and not falling within the scope of the claims. Fig. 7 is a schematic block diagram for explaining a changeover of the error filter that is used in binarizing process 36 shown in Fig. 4. The process shown in Fig. 7 can be performed by software or hardware. Referring to Fig. 7, the type of data J[n] being input is discriminated by block 703 as between natural image and non-natural image (artificial image, e.g., a graphic object or string), and after that an appropriate binarizing process is performed in accordance with the discrimination performed by the block 703. 710 and 720 are adders. 711 and 721 are binarizers using a threshold level. 712 and 722 are error filters for diffusing an error determined by subtracters 713 and 723, which subtract the binarization results from the signals output by adders 710 and 720, respectively. Blocks 710 to 713 include the error filter 712, in which the number of picture elements to which error is diffused, is small. In this example, the error filter 712 comprises four coefficients all, a21, a22 and a23. Blocks 720 to 723 include the error filter 722, in which the number of picture-elements to which error is diffused, is larger. In this embodiment, for example, the error filter 722 comprises twelve coefficients b11, b12, b21, b22, b23, b24, b25, b31, b32, b33, b34 and b35. A sharpened image (in which the high frequency component of the image is relatively large) and a smoothed image (in which the high frequency component of the image is relatively small) are selectively obtained by adjusting the coefficients a11, a21, a22, and a23 and b11, b12, b21, b22, b23, b24, b25, b31, b32, b33, and b34. The printing data 37 is produced in accordance with the type of input image as mentioned above. The preview function of the present example will now be explained. This preview function is a modification of the preview function shown in Fig. 5. Fig. 8 is a schematic diagram of a part of the present example. Referring to Fig. 8, 810 is color image data corresponding to the color image data 531 shown in Fig. 5. The color image data 810 comprises three objects: image file name 811, a string object (e.g., text) 812, and a graphic object 813. These objects are processed respectively by modules 830, 840 and 850 comprised in preview function 820, and are written into a bit map memory 870 that corresponds to element 551 shown in Fig. 5. The modules 830, 840 and 850 produce a proper image by using functions 861, 862 and 863 provided in printer driver 860 that corresponds to element 573 shown in Fig. 5. These functions 861, 862 and 863 perform a binarizing process by using the error diffusion method. The error filters used in the functions 861, 862 and 863 are different from each other, as described in connection with Fig. 7. It will be appreciated that the error filters used by functions 861, 862 and 863, respectively, differ from each other in a manner that corresponds to the differences between the error filters used to process natural image, text, and graphics objects, respectively, for printing. For example, just as the natural-image error filter shown at the bottom of Fig. 7 has more, but smaller, coefficients than the text/graphics filter shown above it, so the preview error filter used for natural image will have more, but smaller, coefficients than the preview error filters used with text and with graphics; especially, the preview error filters for text and for graphics are quite similar. In general, if the size of a given preview error filter is small compared to the other preview error filters, then the corresponding output error filter will also be small, compared to the other output error filters. The module 830 develops the image object on the bit map memory 870. The module 830 transmits image data stored in the file buffer 832 by using function 861, called by a dynamic link 834. The module 840 develops the string object 812 on the bit map memory 870. The module 840 transmits the string object 812 to a string buffer 841 and a color designating field 842. The text developing function 843 develops the string object using font data 845, and binarizes the developed image by using the function 862, called by a dynamic link 844. The module 850 develops the graphic object 813 on the bit map memory 870. The module 850 transmits the graphic object 813 to a graphic object buffer 851 and a color designating field 852. The graphic developing function 853 develops the graphic object 813 and binarizes the developed image by using the function 863, called by a dynamic link 854. The present example, thus, automatically distinguishes among different kinds of image data and processes each kind using the most suitable of the several available error filters. Similarly, the error filter chosen for use in preparing the preview image, is also selected according to the kind of image data. As a result, the preview image provides an accurate representation of the colors that will actually be printed. <First Embodiment> The first embodiment will now be explained using Fig. 9 and Fig. 10. In this embodiment, a preview image is produced based on the difference in appearance between a printed image and a monitor image. Referring to Fig. 9, 901 is original image data having a resolution of 400 dpi. 911 is a resolution conversion process for converting the original image data to a resolution of 360 dpi, which is equal to that of the printer 120. 912 is a dither process for binarizing the printer-resolution image data output by the resolution conversion process 911. The dither process in this embodiment uses a 4x4 threshold matrix. 913 is a printed image produced by printer 120. 921 is a resolution conversion process for converting the resolution 400 dpi of the original image data 901 in accordance with a given instruction. 923 is a tonal level limiting process for limiting tonal level in accordance with a given instruction. 924 is a displayed image on the display monitor 115. 930 is a preview configuration process for controlling the resolution conversion process 921 and the tonal level limiting process 923 in accordance with an operator's instruction. The chart shown in the preview configuration process 930 is a display on the host computer 410. Thus, the operator can enter an instruction for selecting resolution data, and/or data relative to the tonal level limiting process, to the computer 410 while viewing that display. In the above explained embodiment, the resolution of the printed image 913 is less than 360 dpi (and possibly as low as 90 dpi) because the image data has been dither processed using a 4 x 4 threshold matrix, as mentioned above. The guaranteed number of tonal levels of the printed image 913 is 16 because the 4 x 4 threshold matrix is used. Therefore, the resolution of the original image data 901 is reduced to, e.g., 90 dpi, while the tonal level limiting process 923 limits the tonal levels of the original image data to, e.g., 16 levels in this embodiment (this limit may be termed a tonal limiting number"). The image processed by the resolution conversion process 921 and the tonal level limiting process 923 may have an appearance that is similar in some ways to an appearance of the printed image 913, but there will be a difference because of a response characteristic to spatial frequency. The reason for this mismatch is that the image processed by the resolution conversion process 911 includes a frequency component that is higher than can be expressed 90 dpi. Therefore, in this embodiment, the preview configuration process 930 is provided to control the resolution conversion process 921 and the tonal level limiting process 923. An operation of the configuration process 930 is now explained, using Fig. 10. Fig. 10 is a flowchart for explaining the operation of the configuration process 930. S100 is an initializing step for initializing the operation of the tonal level limiting process. S110 is a step for designating the kind of object to be processed (as, a natural or halftone image, characters, or drawings). Sill is a step for discriminating whether a level of image data that corresponds to the object designated is within a predetermined range. If so, the flow continues to step S120, and if not, returns to step S110. In step S120, a resolution is designated. In step S121, a determination is made as to whether that resolution is within a predetermined range. If so, the flow proceeds to a step S130 and, if not, returns to step S110. Step S130 is a step for inputting the tonal limit number designated by the operator. Step S131 is a step for discriminating whether the tonal limited number is within a predetermined range. If so, the flow continues to step S140 and, if not, returns to step S110. Step S140 is a step for performing resolution conversion by process 921 in accordance with the resolution designated in step S120. S150 is a step for performing tonal limiting by process 923 in accordance with the designated tonal limit number. The tonal limiting process 923 performs its process by linearly quantizing color image data which has been resolution-converted, and outputs quantized color image data to the monitor 115. As described above, this results in a preview image similar to the printed image that will be obtained if printing is ordered. <Second Embodiment> The second embodiment will be explained using Fig. 11, Fig. 12 and Fig. 13. In the first embodiment, the resolution conversion process 921 and the tonal level limiting process 923 are controlled in accordance with instructions entered by the operator. However, in the present embodiment, the resolution conversion process 921 and the tonal level limiting process 923 are controlled automatically. Before description of the structure of the present embodiment, some comments on generation of quantization errors by the binarizing process will useful. Fig. 11 is a chart showing quantization errors generated by the binarizing process. In Fig. 11 "O" represents the origin. The X axis and Y axis represent the horizontal direction and vertical direction in the image being binarized, respectively. The qE axis represents a quantization error magnitude. In this example, the image to be binarized has edges at the borders between A and B and between B and C. (A, B, C are only symbols for easy understanding.) There are some points whose quantization errors are nearly zero in zone A and in zone C. There are no such points, however, in zone B. The visual impressions that will be made by the binarized image in zone A, zone C and zone B are different from each other. Therefore, in the present embodiment, the preview image is produced by the following steps. (1) A quantization error that is generated by the binarizing process in each area of the image, is detected. (2) The preview image is produced based on the detected quantization error. Fig. 12 is a schematic diagram of a second embodiment, for explaining the relation between the printed image and the monitor image (in this Figure, those elements that operate like elements shown in Fig. 9 have the same number as in Fig. 9). 1231 is a quantization error calculating device, for calculating a quantization error for each area of the image to be processed. 1232 is a preview image parameter producing device for producing a parameter that controls the resolution conversion process 1221 and the tonal level limiting process 1223. The preview image parameter serves to reduce the number of tonal levels of the image when the quantization error is large. This is done because if the quantization error is large, the impression made by the binarized image is not smooth. The operation of the preview image parameter producing device 1232 is now explained using Fig. 13. Fig. 13 is a flowchart for explaining the operation of the parameter producing device 1232. S1310 is an initializing step for initializing a parameter to be output by the producing device 1232. S1320 is a step for calculating a quantization error for each area of the image when the resolution conversion process 921 and the tonal limiting process 923 operate in accordance with the parameter value set initially in by step S1310. S1330 is a step for comparing the quantization error calculated in step S1320 and the quantization error calculated by the device 1231 for each area. S1340 is a step for discriminating whether the parameter previously used in step S1320 is good or not, in accordance with a comparison result obtained in step S1330. If the existing parameter is satisfactory, the flow continues to step S1360, and if it is not, the flow proceeds to step S1350. S1350 is a step for shifting the value of the parameter to be used in step S1320. After step S1350, the flow returns to step S1320. S1360 is a step for controlling the resolution conversion process 1221 and the tonal level limiting process 1223 in accordance with the parameter that is discriminated as "good" in step S1340. In this manner, the resolution conversion process 1221 and the tonal level limiting process 1223 are automatically controlled in accordance with the characteristics of the image, so that a more precise preview image is obtained. In this embodiment, resolution and tonal limit number are both controlled, but either one may be used for this purpose without the other, in the present invention. Also, the present invention can be applied not only to a color printer but also to a monochrome printer. In addition, this invention can be applied not only to a color monitor but also to a monochrome monitor. Moreover, this invention is applicable not only to systems implemented using computers and software, but also to hardware systems. The quantization error treated in the manner described above may comprise errors produced by a dither method or by an error diffusion method, or any other quantization error. According to the present invention, a precise preview image can be obtained because the preview image is produced based on quantization errors produced in quantizing the image data. The individual components shown in outline or designated by blocks in the drawings are all well known in the pertinent arts, and their specific construction and operation are not critical to the operation or best mode for carrying out the invention. While the present invention has been described with respect to what are at present the preferred embodiments, it is to be understood that the invention is not limited to those embodiments. To the contrary, the invention includes various modifications and equivalent arrangements included within the scope of the appended claims. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims: An image processing method for obtaining a preview image (924) corresponding to an output image to be produced, based on given image data (901), by an image producing device (120), the method comprising the steps of: receiving the image data; and performing preview processing corresponding to the image data and generating said preview image corresponding to an output image to be produced; characterised in that : said preview processing includes a resolution conversion process (921, 1221) and a tonal level limiting process (923, 1223) corresponding to quantization processing (912) that would be carried out by the image producing device (120) such that the preview image expresses a quantization error that would arise due to the quantization processing (912). An image processing method according to claim 1, wherein received image data is color image data. An image processing method according to any preceding claim, wherein the image producing device (120) is a color image producing device. An image processing method according to any preceding claim, wherein the quantization process comprises a binarization process in the case that the image producing device (120) is a color binary-value printer. An image processing method according to any preceding claim, wherein the image producing device (120) is a multi-value printer. An image processing method according to any one of the preceding claims, wherein a resolution conversion parameter and/or a tonal level limiting process parameter are input manually. An image processing method according to any one of the preceding claims further comprising outputting image data for the preview image to a monitor (115). An image processing apparatus for obtaining a preview image of the output image that an image producing device (120) will produce for given image data (901), the apparatus comprising: input means for receiving the image data; preview processing means for processing the image data and generating said preview image corresponding to an output image to be produced; characterised by : said preview processing means comprising a resolution conversion means and a tonal level limiting means for carrying out a process corresponding to quantization processing that would be carried out by the image producing device, such that the preview image expresses a quantization error that would arise due to the quantization processing. An image processing apparatus according to claim 8, wherein the input means is arranged to receive color image data. An image processing apparatus according to claim 8 or claim 9, comprising as said image producing device (120) a color image producing device. An image processing apparatus according to any one of claims 8 to 10, wherein the quantization process comprises a binarization process in the case that the image producing device (120) is a color binary-value printer. An image processing apparatus according to any one of claims 8 to 11, comprising as the image producing device (120) a multi-value printer. An image processing apparatus according to any one of claims 8 to 12, comprising manual information input means for inputting a resolution conversion parameter and/or a tonal level limiting process parameter. An image processing apparatus according to any one of claims 8 to 13, further comprising output means for outputting the preview processed image data to a monitor (115) to enable display of the preview image. A storage medium storing instructions for controlling a processor to implement the method of any one of claims 1 to 7. Processor implementable instructions for controlling a processor to implement the method of any one of claims 1 to 7.

Inventors: NISHIKAWA NAOYUKI, NISHIKAWA, NAOYUKI, NISHIKAWA, NAOYUKI, C/O CANON K.K.
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1995
Main classification: H04N1
IPC codes: G06T1, G06T5, B41J2, H04N1, G09G5