Abstract:
Soft-edge processing of a video color image is effected in accordance with a stylus nib pressure. Initially, a display circle unit is determined having center coordinance corresponding to display coordinance of a stylus and having a radius corresponding to a nib pressure of the nib stylus. A display region is set encompassing the display circle unit, the display region having a plurality of pixels. A distance from each pixel located outside the circle unit within the display region to the circle unit is determined. This distance is then used as the basis for determining a Z value of the pixel which is indicative of the relative illumination of the pixel to effect soft-edge processing of the image.

Description:
This application is a Continuation of now abandoned application Ser. No. 07/638,436, filed Jan. 7, 1991. 
    
    
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to improvements in an apparatus and method for editing and creating video images, and more particularly, to an apparatus and method for editing and creating video images which is capable of effecting a high quality and high speed real-time adjustment of a nib diameter in accordance with an applied pressure of a stylus pen and effecting soft edge processing corresponding to the nib diameter. 
     2. Description of the Prior Art 
     A general conventional apparatus for editing and creating video images includes, among other things, an input device composed of a stylus pen, tablet, keyboard and other such input components, an operation processor for processing various data signals from the input device, and a display device. An image drawn on the table by an operator using the stylus pen is displayed in real time on the display device. 
     In such an apparatus for editing and creating video images, however, the operator must usually select a pen of proper diameter from among specified stylus pens, and draw on the tablet using the selected pen. In actuality, only a few stylus pens are available, and it is difficult to contact the pen on the tablet in a sufficiently delicate manner to obtain precise corrected works having accurate and smooth features. 
     In the outer periphery of a painted image drawn using an actual paint brush, a spontaneous soft boundary (hereinafter called a soft edge) is usually formed by a delicate mixing of the brush color and the paper color. In the apparatus for editing and creating video images using a stylus pen, it is indispensable to subject the video signal to so-called soft edge processing in order to realize a soft edge in the video image. 
     A soft edge processing method has been developed for this kind of apparatus for editing and creating video images, in which the touch of the soft edge processing itself is varied by the stylus pen pressure and fed back (e.g., Japanese Laid-open Patent No. 62-35980). 
     Such a soft edge processing method can simulate existing painting tools (brushes, pens, etc.) to a certain extent, and realize various pen touches of painting tools in a nearly satisfactory manner. 
     Generally, when painting a drawing or correcting a drawing, however, in order to more precisely realize delicate pen touches of painting tools such as brushes, it is more important to vary the diameter of the nib depending on the pressure applied by the stylus pen, rather than to vary the degree of soft edge processing itself by the stylus pen pressure and to feed it back. 
     In the actual world of painting, even when the same brush and same color are used, the touch (pressure) of the brush varies significantly depending on the medium that is painted (for example, the pressure applied when painting on paper and that when painting on cloth differs by about two to one). Therefore, in order to simulate the brush perfectly, it is necessary to effect processing in harmony with a change of nib diameter depending on the nib pressure and the material to be painted on, aside from the change of the degree of soft edge processing. In the conventional soft edge processing method of varying only the degree of the soft edge processing according the stylus pressure and feeding it back, the painting tools cannot be simulated completely, and delicate touches of the painting tools cannot be reliably reproduced. 
     Further, in the conventional soft edge processing method, it is extremely difficult to process the soft edge following a fast moving speed of the stylus pen, and hence it is necessary to restrict the pen moving speed within a certain speed or to use an expensive electronic device. 
     SUMMARY OF THE INVENTION 
     The present invention is intended to solve the convention problems of (1) the difficulty of obtaining a desired pen touch and making fine corrections smoothly since the number of selectable and usable stylus pens is limited, (2) the difficulty of realizing perfect simulation of painting tools and a perfect touch of a brush, and (3) the difficulty of increasing soft edge processing and realizing a soft edge as the moving speed of the stylus pen increases. An object is hence to present an apparatus for editing and creating video images and a method for editing and creating video images capable of high quality painting in real time on a display device without extensive use of expensive devices. 
     The present inventor closely observed the painting works and paint correction works of artists, and found that the artists paint and correct fine parts and bold parts skillfully by using various directions and intensities (pressures) of the brushes depending on actual painting materials. In particular, painting and correction are often done while varying the intensity of the brush (brush pressure). Therefore in editing and creating electronic images, it is found that the diameter of the brush (the thickness of the line of the painted image) should be determined in the first place depending on the stylus pressure, rather than directly determining the degree of soft edge processing (density) of the painting from the stylus pressure. That is, whether the natural painted image can be realized in an electronic apparatus or not is found to depend on how the diameter of the stylus varies depending on the stylus pressure, following the moving speed of the stylus pen, while maintaining the soft edge of the stylus nib. 
     The invention is developed on the basis of the results of observation and analysis, and is fundamentally designed to feed back the pen nib pressure from the stylus pen to an operation processor, varying the nib diameter smoothly, following up the speed of the stylus pen operated by the operator, and processing the painted data signal by painting size variable type anti-aliasing. 
     More particularly, the apparatus of the invention disclosed herein is an apparatus for editing and creating video images including an input device such as digitizer, an operation processor composed of a painting data memory and frame memory, and a display device. The operation processor includes a main processor for controlling the operation of the apparatus; a painting front-end processor for calculating interpolation values by processing pen coordinate values and pen nib pressure values from the input device when executing a command relating to pen painting, and delivering calculated values composed of center coordinate values of a circle painting unit and a radius &#34;b&#34; of the circle unit; a painting address operation processor for supplying addresses to the frame memory corresponding to the coordinate values of the input device; a Z-operation processor for calculating the distance &#34;a&#34; extending from the pixel address integer value Pn calculated by the painting address operation processor to the center value of the circle painting unit and the difference &#34;c&#34; between this distance &#34;a&#34; and the painting radius &#34;b&#34;, and delivering the calculated values Z corresponding to the painting radius i.e. the nib diameter, by referring to a table with the calculated value &#34;c&#34;; and an anti-aliasing processor for receiving the calculated value Z from the Z-operation processor, mixing the original painting data from the frame memory and the painting data from the painting data memory on the basis of the calculated value Z, and feeding the same to the frame memory. 
     The invention disclosed herein is also a method for soft edge processing of the painting drawn on the digitizer and displaying on a display device by means of an apparatus for editing and creating video images including an input device, operation processor and display device, in which the pen coordinate values and pen stylus pressure values entered from the digitizer are arithmetically processed to calculate the individual interpolation values, the calculated value &#34;c&#34; proportional to the pen nib pressure is calculated from the interpolation values, the calculated value Z corresponding to the painting radius is calculated from this calculated value &#34;c&#34; by referring to a table, and the anti-aliasing processing proportional to the calculated value Z is executed on the painting data from the frame memory and painting data memory, thereby making it possible to a paint picture of high quality in real time. 
     The coordinate data and nib pressure data of the stylus pen entered from the input device into the operation processor are individually exchanged with the interpolation values in the painting front-end processor. The calculated interpolation values are fed into the Z-operation processor, and the calculated value &#34;c&#34; corresponding to the nib pressure is calculated, and by using this calculated value &#34;c&#34;, the value of Z is calculated corresponding to the painting radius by referring to the table. 
     The calculated value Z is fed into the anti-aliasing processor, and according to this calculated value Z (that is, the value corresponding to the nib diameter proportional to the pen nib pressure), the original painting data from the frame memory and the painting data from the painting data memory are subjected to anti-aliasing processing and stored in the frame memory. 
     The painting display data after anti-aliasing processing is fed from the frame memory into the display device, and a high quality picture after anti-aliasing processing depending on the nib diameter corresponding to the pen nib pressure is realized in real time. 
     Each processing in this operation processor is executed on the whole area specified by the painting points, and is overlapped on every painting point. 
     In the invention, from the pen nib pressure value and pen coordinate values entered from the digitizer, the interpolation values are determined by the painting front-end processor, and every painting point designated by the interpolation values is processed and overlapped sequentially. As a result, if the processing capacity of the digitizer is slow, the coordinate data and pen nib pressure data are not discontinuous, and the movement of the stylus pen can be simulated smoothly and securely. 
     Further, in the invention, in the Z-operation processor, the calculated value &#34;c&#34; corresponding to the nib diameter which is proportional to the pen nib pressure is determined, and the Z-value is calculated by referring to a table on the basis of the calculated value &#34;c&#34;, and the same is delivered to the anti-aliasing processor. As a result, the Z-value can be determined by preparing a Z-value table possessing a proper &#34;c&#34; value range, and anti-aliasing processing following the movement of the stylus pen is possible. 
     Furthermore, by properly varying the content of the Z-value table, the degree of the anti-aliasing processing may be changed easily, and by properly selecting the width of the value of &#34;c&#34;, the range of anti-aliasing processing may be varied. 
     In addition, in the invention, since the nib diameter is changed depending on the pen nib pressure and the anti-aliasing processing is carried out properly on the basis thereof, as compared with the case of direct soft edge processing from the pen nib pressure as in the prior art, a more delicate brush touch may be realized, and a painting picture of higher quality may be obtained. 
     Thus, according to the invention, without using expensive devices, painting images of extremely high quality may be realized in real time, and superb practical effects are brought about. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     Other and further objects, features, and advantages of the invention will be more explicit from the following detailed description taken with reference to the drawings wherein: 
     FIG. 1 is a block diagram showing the basic composition of an apparatus for editing and creating video images according to the invention; 
     FIG. 2 is a schematic explanatory diagram of interpolation processing in a painting front-end processor; 
     FIG. 3 is an explanatory diagram of the content of arithmetic processing of a Z-operation processor; 
     FIG. 4 is an explanatory diagram showing coordinate points in a real number operation; 
     FIG. 5(a) shows an example of a Z-value reference graph in the Z-operation processor, FIG. 5(b) and FIG. 5(c) are explanatory diagrams of data distribution in anti-aliasing processing and an anti-aliasing processing range; 
     FIG. 6 is a diagram showing the distribution of Z-values in continuous painting processing; 
     FIG. 7 is a block diagram showing an internal composition of the Z-operation processor; 
     FIG. 8 is a flow chart for explaining the operation of a main processor; 
     FIGS. 9(a) and (b) are graphs showing the state of change of contribution rate Z depending on nib pressure; 
     FIGS. 10(a) and (b) are diagrams showing a state of interpolation; 
     FIG. 11 is a flow chart for explaining the painting operation according to the invention; 
     FIG. 12 is a graph showing the stored contents of a Table Tz; 
     FIG. 13 is a graph showing a distribution of Zt and Za; and 
     FIG. 14 (consisting of sub-parts (a)-(f)) is a diagram showing a relation between a radius b and a Z-value table. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Now referring to FIGS. 1 to 6, preferred embodiments of the invention are described below. 
     FIG. 1 is a block diagram showing the composition of an apparatus for editing and creating video images according to the invention. The apparatus includes an input device (digitizer) 1 composed of a stylus pen, a keyboard and a touch tablet, a main processor 2, a painting front-end processor 3, a painting address operation processor 4, a Z-operation processor 5, a frame memory 6, an anti-aliasing processor 7, a painting data memory 8, and an image display device 9. 
     Pen coordinate values based on a coordinate system of the digitizer 1 and pen nib pressure values of the stylus pen operated by an operator are repeatedly output at specific time intervals from the digitizer 1. The thus output data are fed to the main processor 2 and the painting front-end processor 3. The pen nib pressure value denotes the manual pressure applied to the stylus pen by the operator. 
     The main processor 3 processes the pen coordinate values from the digitizer 1, interprets commands from the content of a menu, and sets data corresponding to the commands to the processors 3, 4. When executing a command relating to pen painting, in order to achieve a rapid operation, the pen coordinate values are processed in the painting front-end processor 3. In the main processor 2, cursor display, keyboard sensing, and other processings are carried out. 
     The painting front-end processor 3 is responsible for processing the pen coordinate values X, Y and pen nib pressure W when executing commands relating to pen painting as mentioned above. For example, with coordinates X, Y as a center, a circle painting unit is determined having a radius &#34;b&#34; which is proportional to the nib pressure W. 
     In a case where movement of the stylus pen by the operator is too fast, it is difficult to sufficiently trace the pen trajectory from the coordinate outputs of the digitizer 1 due to the limited processing speed of the digitizer 1. The output coordinate data are discrete rather than continuous, thus creating discontinuities in the trace. 
     To avoid such discontinuities, the processor 3 effects so-called interpolation processing of the input data. 
     FIG. 2 shows an example of the interpolation processing, in which the interpolated values (Xi, Yi, Wi) are calculated from the input pen coordinate data (Xo, Yo), (X1, Y1), and the input pen nib pressure data (Wo), (W1) generated by the digitizer 1. As a result of such interpolation processing, the discontinuities of the coordinate data from the digitizer 1 are removed, thus enabling the simulation of a smooth stylus pen movement. Such simulation is preferably executed by interpolation at a 1/5 pitch of the radius of the circle painting unit. 
     The calculated interpolation values (Xi, Yi, Wi) expressing the center coordinates and radius of each circle painting unit extending along a trace, are output to the painting address operation processor 4 and the Z-operation processor 5. 
     The painting address processor 4 supplies a so-called DMA address to the frame memory 6 corresponding to the coordinate data of the digitizer 1. This DMA address is generated according to coordinates defining a square region which encompasses a circle painting unit and which is centered on the center coordinates of the circle painting unit. 
     The processor 4 calculates an address integer data Pn corresponding to a pixel address on the image display device 9, and feeds the data Pn to the Z-operation processor 5. The address integer data Pn is generated according to a scanning of the square region. 
     The Z-operation processor 5 calculates, as shown in FIG. 3, the center coordinate values of the circle painting unit relative to corresponding digitizer coordinates, from the address integer data Pn value received from the painting address operation processor 4, and calculates the distance &#34;a&#34; from the point of the Pn value to the circle center specified by the digitizer coordinates, and also calculates the difference &#34;c=a-b&#34; between the distance &#34;a&#34; and the painting radius &#34;b&#34;. In addition, the Z-operation processor 5 applies the calculated &#34;c&#34; value to the graphical relation shown in FIG. 5(a) to determine the Z-value corresponding to the painting radius, and delivers the calculated value Z to the anti-aliasing processor 7. 
     The operation unit of the Z-operation processor 5 has a so-called pipe-line structure and performs real time processings. 
     Three to five types of reference graphs is shown in FIG. 5(a) are prepared, each having a necessary width for the range of the &#34;c&#34; value (for example, 0≦c≦200, corresponding to types of a stylus, such as brush, pencil, and airbrush). The type of reference graph is selected by the operator at the command level operation, so that the degree of soft edge processing may be varied. 
     In the invention, as mentioned above, the Z-operation processor 5 calculates the &#34;c&#34; value as shown in FIG. 3 corresponding to the nib pressure, and a Z-value corresponding to the &#34;c&#34; value is then obtained by referring to a prepared graph. Therefore, it is sufficient to prepare a set of Z-value graphs corresponding to &#34;c&#34; value ranges for each type of stylus (for example, approximately 0≦c≦200). By providing a set of reference graphs, soft edge processing of the brush and pen diameter, tracing the movement of the stylus pen, can be realized without using expensive parts. 
     To the contrary, if it is attempted to realize the same function as the Z-operation processor 5 of the invention using a reference graph only (i.e. without calculating the &#34;c&#34; value), after preliminarily writing the Z-values as image data in a memory by preparing all possible data of the radius &#34;b&#34; and the center coordinates at each circle painting unit, the pattern image data of the Z-values become enormous as mentioned below, thus making realization practically impossible. 
     That is, suppose all ranges of the radius from 1 to 512 are prepared as patterns and a resolution of 0.25 is set according to the required precision. Z-value patterns corresponding to 512/0.25=2047 types of radii (exclusive of zero radius) are required. Concerning the center coordinates, the resolution of 0.25 is also required for X and Y respectively, and the pattern image data of Z-values must be prepared independently for the designation below the decimal point. Therefore, supposing the resolution in the X, Y coordinates to be 0.25, the number of center coordinates that can be taken is 16 positions as shown in FIG. 4, and in each one of these center coordinates, pattern image data in 2047 different radii are required. 
     As a result, the total number of pattern image data that are required is 2047×16=32752 patterns, and when five types of distribution patterns of the Z-values are prepared, the necessary Z-value pattern image data is 32752×5=163760 types. 
     As a result, the preparation reference data for each possible circle painting unit requires an enormous amount of Z-value pattern image data, thus making realization impractical. 
     Returning now to FIG. 1, the frame memory 6 is a RAM of multiport access structure. The memory 6 delivers the image display data to the image display device 9, and simultaneously performs a read-modify-write operation. 
     The anti-aliasing processor 7 is composed of two multipliers and an adder. The original image data A from the frame memory 6 is multiplied by the Za-value from the Z-operation processor 5, and the painting data B which corresponds to the color of the image and which is output from the painting data memory 8 is multiplied by the Za-value. The resultant data are summed to obtain image display data D reflecting proper soft edge processing by the Z-value corresponding to the nib diameter. The image display data D is fed to the frame memory 6. 
     A Za-value is the 1&#39;s complement of the Za-value, i.e. equals to the 1-Za as described below. The original image pixel A corresponding to the pixel address from the painting address operation processor 4 is read out from the frame memory 6, and at the same time the Z-value, such as Za and Za, corresponding to the address is calculated in the Z-operation processor 5. The thus calculated Z-value is fed to the anti-aliasing processor 7. The result of the processing in the processor 7 is delivered into the frame memory 6 as the image data D, and is written in the same pixel address as the original image data A. 
     FIG. 5(a) graphically shows a distribution of the Z-value corresponding to a distance from the center of the circle painting unit. In case a&lt;b, Z equals a constant value. In case a&gt;R, Z equals zero. 
     FIG. 5(b) shows a state of soft edge processing in the anti-aliasing processor 7. The original image data A and the painting data B are subjected to soft edge processing to be transformed into the data corresponding to the Z-value reference data. As shown in FIG. 5(c), the anti-aliasing processing is down in a range corresponding to the &#34;c&#34; value of the Z-value reference data, with respect to the boundary of the painting range E. 
     This operation is effected in the entire area specified by the painting points (that is, the whole area in the circle painting unit), and the result is overlaid at each painting point interpolated by the painting front-end processor 3, and delivered from the frame memory 6 to the image display device 9. 
     FIG. 6 shows the relation between pen nib pressure, circle painting units and the distribution of Z-values in a continuous painting process. The painting data B and original image data A are subjected to soft edge processing along a smooth Z-value distribution. Smooth painting expressions from thin lines into thick lines, which is characteristic of painting by a brush, are electronically realized. 
     FIG. 7 is a block diagram showing a structural example of the Z-operation processor 5 which calculates the Z-value. The signals expressing the nib pressure W, radius &#34;b&#34;, and center coordinates X, Y received from the painting front-end processor 3 via signal line 11 are latched respectively in nib pressure register 12, radius register 13 and center coordinates register 14. A signal representing the address integer data Pn is received via signal line 15 from the painting address operation processor 4 and applied to an a-calculation operation circuit 5a. The a-calculation operation circuit 5a also receives the center coordinate values X, Y from the center coordinates register 14. The a-calculation operation circuit 5a determines the distance &#34;a&#34; from the center coordinates X, Y to the coordinates corresponding to the pixel address Pn. 
     The outputs from the a-calculation operation circuit 5a and the radius register 13 are applied to a subtraction circuit 5b, and the difference &#34;c=a-b&#34; is calculated. The table Tc (having data corresponding to the graph of FIG. 5(a)) is referred to, and the contribution degree value Zt corresponding to the value of &#34;c&#34; is obtained for each type of brush. The output from the nib pressure register 12 is applied to the table Tz, and the contribution degree value Zp corresponding to the nib pressure W is obtained. 
     The outputs from the tables Tc, Tz are applied to a multiplication circuit 5c. The multiplication circuit 5c obtains the value Za=Zp×Zt. The values of Zp and Zt are in a range of 
     
         0≦Zp≦1 
    
     
         0≦Zt≦1 
    
     and the value Za is also in a range of 
     
         0≦Za≦1. 
    
     The multiplication circuit 5c outputs signals to the anti-aliasing processor 7 expressing this value of Za and the value Za of the complement of Za where 
     
         Za=1-Za. 
    
     In the anti-aliasing processor 7, the intensity of each pixel is calculated as 
     
         D=Za×B+Za×A 
    
     where A is the original brightness, B is the painting data, and D is a new brightness. 
     FIG. 8 is a flow chart showing the operation of the main processor 2. The process starts at step s1. When it is judged that a command execution is for pen painting at step s2, the operation advances to step s3 in which processing by the painting front-end processor 3 is carried out. When the pen speed is fast at step s4, the interpolation processing is carried out at steps s4 and s5. At step s6, the pen coordinates input processing, such as menu input, is effected. At steps s7 and s8, display and input processings of the cursor and keyboard are conducted. The given command is processed at steps s9 and s10, and when the end is designated, the operation is terminated at step s11. 
     FIG. 9(a) is a graph showing changes of the contribution degree value Z. When the nib pressure W is small, the Z-value is expressed by line 21, and as the nib pressure W increases, the Z-value changes as shown in line 22, line 23 and line 24. It is noted that, as shown in FIG. 9(b), the peak value of the contribution degree value Z is constant even though the radius &#34;b&#34; of the painting circle unit changes corresponding to the changes of the nib pressure W. 
     FIG. 10(a) is a graph showing the state of interpolation processing to obtain smooth painting along the movement of the pen 1. The painting front-end processor 3 interpolates between the first circle painting unit 31 and second circle painting unit 32, and sets the intermediate circle painting unit 33. FIG. 10(b) shows the contribution degree value Z corresponding to the circle painting units 31, 32, 33. 
     FIG. 11 is a flow chart showing the above-mentioned painting operation. Starting at step t1, the coordinates X, Y designated by the stylus pen 1 and the nib pressure are read respectively at step t2 and t3. A square region, which includes the circle painting unit corresponding to the coordinate X, Y, the radius &#34;b&#34; and the nib pressure, is determined at step t4. 
     At step t5, a scanning operation starts within the square region. A distance &#34;a&#34; from the center X, Y to a scanned pixel address Pn is calculated at step t6. This value &#34;a&#34; is compared with the value &#34;b&#34; at step t7. 
     In a case where &#34;a&#34; is greater than &#34;b&#34;, the Z-value is obtained from a Z table addressed using the &#34;c&#34; value (a-b) at step t8 and t9. In a case where &#34;a&#34; is less or equal to &#34;b&#34;, the Z-value is fixed to a predetermined constant value at step t10. 
     At step t11, previous data corresponding to the previous pixel intensity is read from the frame memory 6 addressed by the data Pn. In previous data and the Pen data representing characters and colors imparted to the pen 1 are combined to calculate a new data at step t12. The new data is written into the frame memory 6 at the pixel address Pn at step t13. 
     Until the scanning within the square areas is terminated at step t5, steps t5 to t13 are repeated. This scanning process is terminated at step t14. 
     FIG. 12 is a graph showing that the value Zp varies according to a nib pressure W. 
     FIG. 13 is a graph showing that the values Zt and Za vary according to the distance &#34;c&#34;. 
     FIGS. 14(a) through 14(f) show a relationship between the nib pressure W and the Z value table. 
     FIG. 14(a) shows a range of the nib pressure W normalized from 0 to 1. FIG. 14(b) shows a Z distribution when W equals 1. FIG. 14(a) through 14(e) show a Z distribution corresponding to W values of 0.6, 0.3 and 0.2, respectively. In case the nib pressure W is less than 0.3, the radius &#34;b&#34; is assumed to be negative as shown in FIG. 14(f). Z values corresponding to the various W values are obtained from only one Z table. 
     The invention may be embodied in other specific forms without departing from the spirit or essential characteristic thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description and all changes which come within the meaning and the range of equivalency of the claims are therefore intended to be embraced therein.