Apparatus and methods for converting raster illustrated parts images into intelligent vector-layered files

Apparatus and methods for converting raster illustrated parts images into intelligent vector-layered files. The method involves recognizing and removing reference labels from the raster illustrated parts image to produce a reference label scrubbed file. Reference lines are recognized and removed from the reference label scrubbed file to produce a scrubbed file. The scrubbed file includes a reusable base graphic. The scrubbed file is converted to a vector file in which the reusable base graphic is embedded. One or more vector layers are added to the vector file to produce the intelligent vector-layered file. Each vector layer includes vector elements corresponding to one of the recognized reference labels and its one or more reference lines.

COPYRIGHT NOTICE

A portion of the disclosure of this document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the U. S. Patent and Trademark Office patent files or records, but otherwise the copyright owner reserves all copyright rights whatsoever.

FIELD OF THE INVENTION

The present invention relates to raster illustrated part images, and more particularly to apparatus and methods for converting raster illustrated parts images into intelligent vector-layered files.

BACKGROUND OF THE INVENTION

Illustrated parts drawings identify a hierarchy of details or assemblies and parts in a manner showing how the details and parts fit together. Illustrated parts drawings may show multiple details such as the exemplary illustrated parts drawing shown inFIG. 1that includes three details labeled G, H and I. Typically, the details include references or items numbers that are indexes into a parts list where additional information about the assemblies and parts is available. A single illustrated parts drawing can include many details with dozens of item numbers.

It is a common practice to use illustrated parts drawings with task lists. A task list specifies construction or maintenance steps, where each step references one or more the parts on the illustrated parts drawing. For a particular step, a user typically must search the drawing for the parts referenced in the step to view the part and how it relates to other parts. However, searching for the part can be time-consuming and prone to errors, especially as the number of parts contained in the illustrated parts drawing increases.

An existing method of improving the usability of electronic illustrated parts images with task lists is to separate the various images of the illustrated parts drawings for each step of the task list and then identify only the details and parts referenced in that step. In this method, the same base drawing is used repeatedly but with only the relevant, and different, parts being identified each time. This method immediately draws the user's attention to the parts or items on the drawing that are relevant to the current step of the task list. For example,FIG. 2shows an illustrated parts image being used with an exemplary task list. As shown inFIG. 2, the illustrated parts image identifies only the parts or items relevant to or mentioned in the current task list step (i.e., “Remove bolts, washers, and nuts”). Although this method has proved successful for its intended purpose, the cost of manually creating and maintaining numerous slightly modified versions of the same drawing, however, is prohibitively expensive.

Another method of improving the usability of electronic illustrated parts images is to provide an illustrated parts image with one or more intelligent objects. Indeed, existing computer software programs and tools allow for the authoring of intelligent illustrated parts images with intelligent objects and constructs, such as item numbers and locators. By way of example only, an illustrated parts image may be provided with an item number that is disposed at the end of a reference line (e.g., lead line, leader line, arrow, bulleted line, etc.) and that is associated with a link or index to database information about the particular component or part referenced by the item number. Accordingly, a user-click on an item number queries a database and thus allows the user to access database information associated with the item number. As another example, an illustrated parts image may be provided with a locator. As before with item numbers, a locator is also disposed at the end of a reference line. However, a locator is associated with zooming functionality that allows a user to zoom in on a particular portion (e.g., component, part, detail, assembly, etc.) of the illustrated parts drawing with a user-click on the locator. Accordingly, both item numbers and locators allow a user to access additional information by way of a user-click thereon.

However, there are many existing illustrated parts drawings that comprise unintelligent raster images (bitmapped graphics) that do not provide high-level structures, such as text records or graphical primitives. For at least this reason, raster illustrated part images have had very limited functionality in electronic information systems.

SUMMARY OF THE INVENTION

Accordingly, the inventors have recognized a need in the art for devices and methods that improve the usability and functionality of raster illustrated parts images by converting existing raster illustrated parts images into intelligent vector-layered files in a highly accurate, efficient, and automated batch process that requires little to no user intervention.

The present invention is directed to a system and method for converting raster illustrated parts images into intelligent vector-layered files. The method generally involves recognizing and removing reference labels from the raster illustrated parts image to produce a reference label scrubbed file. Reference lines are recognized and removed from the reference label scrubbed file to produce a scrubbed file, which includes a reusable base graphic. The scrubbed file is converted to a vector file in which the reusable base graphic is embedded as a bitmap. One or more vector layers are added to the vector file to produce an intelligent vector-layered file. Each vector layer includes vector elements corresponding to one of the recognized reference labels and its one or more reference lines.

Corresponding reference characters indicate corresponding features throughout the drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring toFIG. 3, there is shown a control system10in accordance with a preferred embodiment of the present invention. Generally, the system10converts raster illustrated parts images11into one or more intelligent vector-layered files13in a substantially automated batch process. Each intelligent vector-layered file13includes a reusable base graphic and a vector layer for each item number and locator. Each vector layer also includes the reference line(s) associated with the corresponding item number or locator.

For ease of identification and description and not for purposes of limitation, the term “reference label” as used herein shall be construed to include both item numbers and locators. In addition, the term “reference line” as used herein shall be construed to include any of a wide range of lines regardless of whether the line has an end designator, including but not limited to arrows, lead lines, leader lines, bulleted lines (i.e., lines with bullets as end designators), among others.

The vector layers are preferably encoded in a file format that is compatible with existing electronic or drawing management systems to allow authors to link vector layers to related steps of a task list. At runtime, the vector layers can be activated (i.e., made visible) or deactivated (i.e., made invisible) so that the items relevant to the current step are identified on the reusable base graphic. SeeFIGS. 4A and 4B. In other words, the reusable base graphic and various vector layers comprise an intelligent graphic, the display of which varies depending upon the content or particular step for which the graphics are being displayed.

Referring back toFIG. 3, the system10includes a suitable processing element12for performing the various operations required by the present invention. The processing element12is typically comprised of a combination of hardware (e.g., one or more microprocessors, other processing devices) and software that is stored by memory and executed by the hardware. In the illustrated embodiment, the processor12executes a detail art grouping and separation module14, a text recognition and erasure module16, a line recognition and erasure module18, a graphic normalizer module20, a vector file conversion module22, and a vector layer builder and inserter module24. However, it should be understood that the processing element12can be comprised of other combinations of hardware, software, firmware or the like so long as the resulting combination is capable of implementing the various operations required for converting raster illustrated images into one or more intelligent vector-layered files.

The system10also includes memory which may take the form of any suitable computer readable storage device. For example, the memory may comprise read only memory (ROM), random access memory (RAM), video memory (VRAM), hard disk, floppy diskette, compact disc (CD), an optical disk, magnetic tape, a combination thereof, etc. The memory may comprise computer readable media for storing such items as program code, software packages, programs, algorithms, information, data, files, databases, applications, among other things.

In the embodiment shown inFIG. 3, the system10includes the detail art grouping and separation module14, the text recognition and erasure module16, the line recognition and erasure module18, the graphic normalizer module20, the vector file conversion module22, and the vector layer builder and inserter module24. The plurality of modules14through24may be embodied in computer-readable program code stored in one or more computer-readable storage media operatively associated with the system10.

It is to be understood, however, that the computer readable program code described herein can be conventionally programmed using any of a wide range of suitable computer readable programming languages that are now known in the art or that may be developed in the future. It is also to be understood that the computer readable program code described herein can include one or more functions, routines, subfunctions, and subroutines, and need not be combined in a single package but may instead be embodied in separate components. In addition, the computer readable program code may be a stand-alone application, or may be a plug-in module for an existing application and/or operating system. Alternatively, the computer readable program code may be integrated into an application or operating system. In yet another embodiment, the computer readable program code may reside at one or more network devices (not shown), such as an administrator terminal, a server, etc.

Although the present invention is described with the various modules14through24having a direct effect on and direct control of the system10, it should be understood that it is the instructions generated by the execution of the programs14through24by the processing element12, and the subsequent implementation of such instructions by the processing element12, that have direct effect on and direct control of the system10.

The system10further includes data and information specific to the set of raster illustrated parts images being converted. As shown, the system10includes a plurality of specially built textual character sets25,26, and27, one set for the item numbers, one set for the detail labels, and one set for the locator labels. As explained in greater detail below, the character sets25,26, and27are used by the processing element12during detail art grouping and separation and during text recognition.

A preferred method30implemented by the system10of the present invention is illustrated in simplified flow chart form inFIGS. 5A and 5B.FIG. 6is a process flow diagram showing various files used and/or created during the method30.

As shown inFIG. 5A, step32of method30comprises inputting one or more files132(FIG. 6) from the illustrated parts document11into the system10. By way of example only, the files132input at step32may comprise uncompressed TIFF (tagged image file format) files having various resolutions (e.g., 300 dpi (dots per inch) to 700 dpi) and various color depths (e.g., color depth of 8 or 256 colors, etc.).

At step34(FIG. 5A), the system10executes the graphic normalizer module20(FIG. 3) to reformat and standardize each input file132(FIG. 6) to the same resolution, color depth, and compression. By way of example only, the files134(FIG. 6) normalized by the system10at step34may be compressed and converted to 300 dpi and 1 bit color (monochrome).

Step36(FIG. 5A) comprises a detail art grouping and separation process during which the system10executes the module14(FIG. 3) to separate the details within the normalized files134(FIG. 6) into individual or detail separated files136. As shown inFIG. 5A, step38involves the system10accessing the character sets25,26, and27and using optical character recognition (ocr) to recognize the text (e.g., detail labels, locator labels, item numbers, etc.) within the raster illustrated parts images contained within the normalized files134. At step38, the system10may, for example, execute optical character recognition (OCR) software, such as Cartouche® OCR computer software from RAF Technology, Inc. of Redmond, Wash.

At step40, the system10stores the recognized text and its corresponding locations. At step42, the recognized text is removed or erased from the raster images within the normalized files134.

At step44, the system10detects and parses the individual details within the raster images of the normalized files134by using geometric proximity-based calculations in conjunction with the detail labels recognized at step38. The system10uses the detail labels as starting points to identify pixels that are connected or grouped together. The system10preferably begins searching above the detail labels for the pixels to be grouped for each detail label. Each collection of grouped pixels may include the artwork forming the corresponding detail and its associated reference lines. Although the system10preferably separates ambiguous images (i.e., images that cannot be separated out with high confidence), the system10also preferably flags the ambiguous images to notify a user (e.g., illustrator, etc.) of the ambiguity.

At step46, the system10stores each detail found at step44in its own file136(FIG. 6). For example, the three details labeled G, H and I inFIG. 1would each be written to their own files at step46as illustrated inFIGS. 8A,8B and8C.

At step48(FIG. 5A), the system10executes an assignment algorithm to assign and write the text recognized at step38to the appropriate detail file136. Text assignment is preferably based upon geometric proximity reasoning (i.e., how close a particular text element is to a detail). The unambiguous text is preferably assigned before ambiguous text (i.e., those that cannot be assigned to a detail with high confidence). Each text element is preferably assigned to the detail to which its corresponding reference line is pointing.

Step50comprises a reference label recognition and erasure process during which the system10executes the module16and optical character recognition computer software. The input for the reference label recognition and erasure process50comprises the detail separated files136. At step50, the system10finds the text corresponding to the reference labels while allowing for at least some deviation in the characters.

During the reference label recognition and erasure process50, the system10first removes all large binary large objects at step52(i.e., the binary large objects that are too large to be characters) so that the same may be ignored during the reference label recognition and erasure process50. Ignoring the large binary large objects substantially improves the system's10processing speed and accuracy during the reference label recognition and erasure process50. As used herein, a “binary large object” (BLOB) is a grouping of all visible pixels that are connected, either horizontally, vertically or diagonally, to one or more other visible pixels in the grouping. A visible pixel is a pixel whose color is different than that of the background color of the image. InFIG. 7, there is shown an illustration of an exemplary raster image fragment in which four binary large objects or blobs have been identified.

Referring back toFIG. 5A, step54comprises character recognition during which the system10runs a character recognition engine on the remaining “small” blobs (i.e., those binary large objects or blobs that are not too large to be characters). During step54, the system10accesses the data within the character sets25,26, and27to locate the characters within the detail separated files136.

At step56, the system10uses the output from the character recognition step54to find reference labels, classifying them as item numbers, locator labels or detail labels.FIG. 9shows the reference labels that were found inFIG. 8Aby the system10.

At step58(FIG. 5A), each recognized reference label and its respective placement or position on the image is captured or stored in an intermediate version of the raster file. The pixels representing each reference label are then erased or scrubbed from the images at step60to produce reference label scrubbed raster files160(FIG. 6).

Referring now toFIG. 5B, step62comprises a line recognition and erasure process during which the system10executes module18to identify, store, and erase the reference lines associated with the reference labels. The input for the line recognition and erasure process62comprises the reference label scrubbed raster files160(FIG. 6).

At step64(FIG. 5B), the system10builds an ordered list of pixel runs from a reference label scrubbed raster file160. As used herein, a “pixel run” is a grouping of pixels that are adjacent horizontally and share the same y-coordinate, as shown inFIG. 10. The initial runs built at step64each have a height of one (1) pixel but may have varying lengths or widths depending on how many adjacent pixels are found. A typical raster image may include tens of thousands of pixel runs.

At step66(FIG. 5B), the pixel runs having identical x-coordinates and adjacent y-coordinates are merged. When a run is merged with another run it is removed from the list of runs. The remaining run is modified so that it is now “thicker” or “taller” than it was before the merge. For example,FIG. 11Ashows fifteen (15) pixel runs, each having a width or length of three (3) pixels and height of one (1) pixel, that can be merged to form the single pixel run of width three (3) pixels and height of fifteen (15) pixels shown inFIG. 11B. Accordingly, horizontal lines are merged runs whose widths exceed their heights while meeting a minimum length requirement, whereas vertical lines are merged runs whose heights exceed their widths while meeting a minimum length requirement. Oblique lines are made of pixel runs that are “staircased”, as shown inFIG. 12.

At step68(FIG. 5B), the system10searches for starting positions for reference lines by searching for pixels in the immediate proximity to where reference labels were recognized at step50. That is, a line start is located by finding a pixel run in the immediate proximity to where a reference label was located. Once found, the pixel run is used as a starting point or seed by the system10to locate adjacent pixel runs that also proceed in the substantially same direction and along a substantially constant slope as the starting pixel run. When the system10cannot locate anymore of such adjacent pixel runs, a determination is made as to whether the collection of pixel runs, now forming a line, is sufficiently long enough for consideration as a reference line. If not, the system10selects another pixel run for use as starting point or seed and builds collection of pixel runs therefrom in the manner just described.

It should be noted that the resolution for the raster illustrated parts images being converted may vary depending on the particular application in which the present invention is being used. Accordingly, the determination of when a collection of pixel runs is long enough for consideration as a reference line will also vary depending on the particular application in which the present invention is being used. By way of example only, a reference line might be required to be at least five to ten percent (5–10%) coverage of the artwork.

In the illustrated embodiment, the system10tests at step70for the presence of an arrowhead at the end of each potential reference line (i.e., the pixel run collections that are sufficiently long enough to be considered a reference line). The system10works backward from a line stopping point and looks for pixel runs forming an arrowhead shape. If an arrowhead is found, the object (i.e., the line and arrowhead) is considered a valid reference line. After a valid reference line is located for a reference label, the system10may continue searching for other reference lines for the reference label because a single reference label may have more than one reference line associated therewith, as shown for the item number435in the detail labeled G inFIG. 1.

If no arrowhead is found for a line at step70, the system10does not consider the line to be a reference line. The system10does, however, continue testing other lines for arrowheads. It should be noted, however, that the system10can also be configured to search for other types of lines, such as lines with bullets instead of arrows, lead lines (i.e., lines without end designators), etc.

At step72, the system10writes one or more location files172(FIG. 6) that contain data and information about the text strings for reference labels as well as their respective image locations. The location files172also contain data and information pertaining to the start and end locations of each reference line. InFIG. 13, there is shown an exemplary location file wherein “IN” refers to Item Number, “LO” refers to Locator, “DL” refers to Detail and “LL” refers to Leader Line.

At step73(FIG. 5B), the system10creates the scrubbed files173(FIG. 6) containing the reusable base graphic(s) (e.g.,FIG. 14) by erasing the pixels forming the reference lines, and accompanying arrowheads from the reference label scrubbed files160. However, the system10looks for pixels adjacent to the reference line in order to avoid erasing pixels that are also part of or overlapping other objects on the image.

At step74(FIG. 5B), the system10executes the module,20(FIG. 3) to reformat and standardize the scrubbed files173(FIG. 6). At step76(FIG. 5B), the system10executes the module22(FIG. 3) to convert the normalized scrubbed files174(FIG. 6) into vector files176that include the reusable base graphics as embedded bitmaps.

By way of example only, the system10may encode the vector files176in a file format called Computer Graphic Metafile (CGM), a widely-used technical illustration format. Alternatively, however, other languages and file formats may be used by the system10to encode the vector files176including, but not limited to, DWG format, document exchange format (DXF), initial graphics exchange specification (IGES) format, among others.

At step78(FIG. 5B), the system10executes the module24(FIG. 3) to create the intelligent vector-layered files178(FIG. 6). That is, the system10reintroduces into the vector files176the reference labels and their associated reference lines as vector elements with each reference label being within a separate vector layer. As used herein, a “vector layer” shall be construed to include one or more vector elements that can be dynamically rendered or hidden at run-time under program control. As evident by comparingFIGS. 8A and 15, the intelligent vector-layered image (FIG. 15) appears substantially identical to the original raster image shown inFIG. 8Awhen all layers of the intelligent vector-layered image are turned on.

During step78(FIG. 5B), the system10accesses the location files172(FIG. 6) and uses the information contained therein to create a separate vector layer for each reference label and its corresponding reference line(s). Accordingly, in the illustrated embodiment, each vector layer contains the lines emanating from the reference label, a polygon for the arrowhead at the end of each line, and a circle (for item numbers) or rectangle (for locator and detail labels) at the location of the reference label. Appropriate vector elements may be used to encapsulate the circle or rectangle so that it becomes a hotspot upon which a user can click to access the corresponding part information or detail. The text strings for the reference labels in the corresponding vector layers are located in substantially identical positions as they were in the original raster illustrated parts image11.

Upon completion of step78, the intelligent vector-layered files178may be saved on a suitable computer readable medium at step80(FIG. 5B). Alternatively, or additionally, the intelligent vector-layered files178may be output at step82, for example, to a graphical display.

The reference labels and their reference lines may also be encoded in extensible markup language (XML). In addition, the intelligent vector-layered files178preferably comprise CGM Version 4 files, which allow for the control of vector layers by existing graphic display software and allow vector layers to be made visible individually or as a group. Alternatively, however, other languages and file formats may be used for the intelligent vector-layered files.

In any event, the intelligent vector-layered files178allow document authors to control what layers are visible and when. Thus, one raster illustrated parts image with tens or hundreds of items may be reused repeatedly while displaying only the items applicable to a particular step in the task list, as shown by comparingFIG. 4A(identifying all reference labels) andFIG. 4B(identifying reference labels relevant to a particular step).

Referring now toFIG. 16A, item number text may be included as an aid when document authors are working with the intelligent vector-layered images. Document authors might need to know what each item number refers to because item number text is typically an index into a database parts table. However, it is usually not necessary for the item number text to be displayed (FIG. 16B) for users viewing the illustrated parts images with an electronic information system because the electronic information system can perform the table lookup and return the results automatically, thus eliminating the need for the users to know the item number text or do the cross-referencing manually.

The system10preferably comprises a batch conversion processor and thus does not require significant human intervention or manual re-authoring of the raster images. Accordingly, the present invention provides a more practical and cost-effective solution to the task of converting raster illustrated parts images to intelligent vector-layered files than the solutions presently recognized in the art, such as manually re-authoring or converting illustrated parts drawings with commercially available tools.

By accurately and quickly converting raster illustrated parts images to intelligent vector-layered files, the present invention dramatically improves the usability of the data within illustrated parts drawings. For example, the intelligent vector-layered files are suitable for integration with other intelligent graphics capabilities, advanced and efficient user interaction, among other functional capabilities.

In addition, the present invention is highly accurate in recognizing reference labels and reference lines from raster images. Indeed, the present invention provides a high quality but inexpensive approach for converting paper and raster illustrated parts drawings into intelligent vector-layered drawings. The present invention also eliminates, or at least reduces, the need for paper-based deliveries of illustrated parts drawings in that paper-based illustrated parts drawings can be scanned and then converted by the present invention to intelligent vector-layered drawings.

It is anticipated that the invention will be applicable to any of a wide range of raster graphics. Accordingly, the specific references to raster illustrated parts images herein should not be construed as limiting the scope of the present invention, as the invention could be applied to convert other raster images to intelligent vector-layered images, including but not limited to assembly instructions for consumer products, assembly instructions in the automotive industry, among others.

The description of the invention is merely exemplary in nature and, thus, variations that do not depart from the substance of the invention are intended to be within the scope of the invention. Such variations are not to be regarded as a departure from the spirit and scope of the invention.