Non-conformance mapping and visualization

Areas of non-conformances in a manufactured object are electronically mapped within a coordinate system of the object. Boundary lines of the areas containing the non-conformances are displayed on a 3-D image of the product. Visualization of the boundary lines of areas containing multiple non-conformances allows tracking of non-conformances, identification of trends in non-conformances and correction of production processes in order to reduce non-conformances.

CROSS REFERENCE TO RELATED APPLICATIONS

BACKGROUND INFORMATION

The present disclosure generally relates to systems for tracking non-conformances in manufactured objects, and deals more particularly with a system for mapping and visualizing areas on the object containing multiple non-conformances.

In production environments, manufactured products may be produced along a line of stations or shops where individualized manufacturing or assembly operations are performed. During these operations, non-conformances may occur in the object for any of a number of reasons. In some cases, non-conformances can be corrected, or future non-conformances can be avoided by tracking the non-conformances and modifying production processes and/or equipment accordingly.

For example, during the production of commercial aircraft, aircraft assemblies are moved from shop-to-shop on a factory floor and assembled according to a predetermined installation plan. Individual non-conformances may be captured and recorded by personnel at each shop, so that personnel in subsequent shops are made generally aware of existing non-conformances, and may take steps to correct them. Currently, shop floor personnel manually record individual non-conformances using non-standard text formatting and/or spreadsheets in which the general location of non-conformances are noted. However, this method is not integrated with other data sources such as part drawings, specifications, etc. and does not record the exact location of the defect in terms of absolute X,Y,Z coordinates that allow a non-conformance to be readily located. Consequently, in some cases, repair work to correct non-conformances is unintentionally covered up, in turn resulting in the need for substantial rework, and associated repair costs. Moreover, the current method does not permit identification of large areas containing multiple non-conformances that require rework.

SUMMARY

The disclosed embodiments provide a system for mapping and visualizing non-conformances on a manufactured object or structure, such as an aircraft, or a group of objects such as a fleet of aircraft. The system provides a user with a dashboard view on electronic display and is capable of processing large amounts of non-conformance records from source data systems. Source data may include aircraft control codes, and installation plans used in the production of aircraft. The dashboard displays an image of the aircraft along with the point locations of non-conformances which are mapped onto the image. The system may include a 3-D visualization program capable of displaying a 3-D image of the aircraft down to the part level, and showing the non-conformance record status, age and location of non-conformances.

The system may employ commercially available dashboard software tools. The system interfaces with these tools and selected data systems to display various types of information required by a user in connection with the production of aircraft or other objects or products. Using the spatial coordinates of each non-conformance, the system generates a 3-D outline or boundary of the shape and size of an area containing multiple non-conformances, including the contours of the area. The 3-D image showing the point locations of each of the non-conformances, and the boundaries of the area containing the non-conformances, is stored for future use in connection with production of other aircraft and tracking non-conformance trends.

According to one disclosed embodiment, a method is provided of mapping and visualizing non-conformances on an object. The method comprises determining a location of each of the non-conformances on the object and using a processor to generate an electronic record containing spatial coordinates of the non-conformances within a 3-D coordinate system. The method further comprises using a processor to calculate boundaries of an area on the object containing the non-conformances based on the spatial coordinates contained in the electronic record, and generating an electronic image of the object, including displaying the boundaries of the area containing the non-conformances to allow visualization of the area containing the non-conformances. Generating the electronic record is performed by generating a 3-D electronic display of the object, selecting points on the 3-D electronic display respectively corresponding to the locations of the non-conformances, and using a programmed computer to convert the points selected on the 3-D electronic display into the spatial coordinates within the 3-D coordinate system. Selecting points on the 3-D electronic display includes moving a cursor to locations on the electronic display corresponding to the locations of the non-conformances on the object, and selecting the locations on the electronic display based on the location of the cursor. Generating includes scanning the object using a feature scanner, using a processor to generate a set of data points defined by X,Y,Z coordinates representing a surface of the object, and using a processor to convert the data points to the 3-D electronic display. Generating the 3-D electronic display may be performed by using a programmed computer to derive a 3-D digital model of the object from a 3-D CAD file and generating the electronic record includes using a processor to compute X,Y,Z coordinates of the points on the 3-D electronic display respectively corresponding to the locations of the non-conformances. Displaying the electronic image with boundaries of the area containing the non-conformances is performed at any of a plurality of stations where work is performed on the object. The 3-D coordinate system is a coordinate system of the object. Determining the location of each of the non-conformances on the object may be performed using a machine vision system to locate each of the non-conformances on the object.

According to another disclosed embodiment, a method is provided of visualizing an area of non-conformances in a manufactured object comprising generating a 3-D electronic display of the object, and selecting points on the 3-D electronic display respectively corresponding to locations of the non-conformances on the object. The method also includes using a programmed computer to convert each of the points on the 3-D electronic display into a set of coordinates defining the locations of the non-conformances within a coordinate system of the object. The method further includes using a programmed computer to calculate boundaries of an area containing the non-conformances based on the set of coordinates, and generating an electronic image of the object, including displaying the boundaries of the area containing the non-conformances to allow a user to visualize the area containing the non-conformances. Generating the 3-D electronic display is performed using a programmed computer to access a 3-D CAD file representing a model of the object, and generating the 3-D electronic display may be performed using a 3-D point cloud generated by scanning the object. Generating the electronic image of the object includes displaying the electronic image in 3-D, and displaying contours of the boundaries. The method may further comprise using a programmed computer to generate an electronic record for the object that contains a set of coordinates, and entering information into the electronic record uniquely identifying the object and types of non-conformances.

According to another disclosed embodiment, a manufacturing method is provided, comprising identifying non-conformances on each of a plurality of substantially identical manufactured objects, and using a programmed computer to generate an electronic record for each of the objects, including entering into each of the records spatial locations of non-conformances on the object. The method further comprises using a processor to map the non-conformances on each of the objects using the electronic records, and displaying an electronic image of one of the objects, including displaying boundary lines of areas containing non-conformances for each of the objects. The method may also include modifying at least one process used to manufacture the objects based on the boundary lines. Displaying the electronic image includes superimposing the boundary lines on the image. The method may also include determining an area containing the non-conformances of all of the objects based on the boundary lines displayed on the image, and reworking the non-conformances within the area. Entering spatial locations of non-conformances on the object includes entering coordinates of each of the non-conformances in a 3-D coordinate system of the object.

According to still another disclosed embodiment, a system is provided for tracking non-conformances of aircraft occurring during production. The system comprises a programmed computer, at least one electronic display coupled with the computer for displaying a 3-D image of the aircraft, and an electronic file accessible by the computer and containing a 3-D model of the aircraft. The system further comprises an input device coupled with the computer and configured to allow a user to input non-conformances based on the 3-D image on the electronic display, and a software program. The software program is accessible by the computer, and includes a routine for mapping the spatial locations of the non-conformances in 3-D, and a routine for calculating boundaries of a 3-D area containing the non-conformances to allow visualization on the display of non-conformances within the boundaries.

DETAILED DESCRIPTION

Referring first toFIG. 1, the disclosed embodiments provide a production non-conformance visualization system20(hereafter “PNVS”) for mapping and visualizing non-conformances on a manufactured object, such as, for example and without limitation, an aircraft32. As used herein “non-conformance” and “non-conformances” refer to parts or areas of the aircraft32that are out-of-tolerance, or exhibit inconsistencies and/or may not meet performance or aesthetic requirements. The PNVS20is capable of both mapping the non-conformances and displaying electronic 3-D images56that show the boundaries of areas58on the aircraft32containing one or more non-conformances, and optionally, the point locations of each of the non-conformances.

Referring now toFIGS. 1 and 2, in one exemplary application, the PNVS20may be employed in connection with a production line22in which aircraft32pass through a series of shops24(FIG. 2) or production stations where various production operations are performed on the aircraft32, such as, without limitation, machining, assembly, testing, painting, etc., according to a predefined aircraft installation plan44. The shops24may be located in a single facility such as a factory (not shown), or in multiple facilities.

Each of the shops24may include one or more electronic display screens30such as a flat screen monitor or tablet (not shown), and other data processing components, including, for example and without limitation, an input device66such as a keyboard or touch screen (both not shown). The shops24are networked with a computer system34and various electronic databases and files, including but not limited to a non-conformance database26, 3-D point cloud files50, 3-D CAD files52, and other data sources48which may comprise information for various models of the aircraft32, quality control records, statistical information, etc.

The computer system34may comprise a central computer system, or a distributed system in which hardware, firmware or software implemented parts of the computer system34are located within one or more of the shops24, or elsewhere. The computer system34includes one or more programmed computers36or similar data processors controlled by one or more software programs38. The computer system34also includes a 3-D display program46capable of displaying electronic 3-D images56on any of the display screens30, and a non-conformance mapping and display program39capable of capturing, mapping and displaying areas of non-conformances on the 3-D images56. As will be discussed later in more detail, the non-conformance mapping and display program39may include a non-conformance capture routine40and a non-conformance display routine42. A commercially available dashboard display program47including a graphical user interface (GUI) may also be employed to provide a dashboard type display on the display screen30.

The computer system34may have access to a variety of data sources48, the aircraft installation plan44, the 3-D display program46, 3-D point cloud files50, 3-D CAD files52and the non-conformance database26for multiple aircraft32, such as an aircraft fleet. The non-conformance database26comprises a plurality of electronic records45, wherein each of the electronic records contains information uniquely identifying a particular aircraft and the spatial coordinates of non-conformances on that aircraft32within the three-dimensional coordinate system64of the aircraft32.

The data sources48may include various types of data generated at each of the shops24, such as quality control data and historical data for a fleet of aircraft, etc. The 3-D CAD files52may include, for example and without limitation, solid models of parts and assemblies for multiple types of aircraft32which may be called up to display various sections and parts of the aircraft32. Alternatively, the computer system34may have access to 3-D point cloud files50that represent surfaces on the aircraft32which are derived from scanning surfaces of the aircraft32using laser scanners62(FIG. 1) or other types of feature capture devices. As will be discussed below, non-conformances may be identified either through visual inspection by personnel in any of the shops24, or by using one or more optional machine vision systems60which may detect non-conformances as well as provide the spatial locations of the non-conformances within the three dimensional coordinate system64of the aircraft32.

As previously discussed, the PNVS20allows a user to map non-conformances on a section of the aircraft32, such as a wing54, and then visualize areas58of the wing54that contain those multiple non-conformances. Referring toFIG. 3, an electronic dashboard display28may be called up by a user on one of the display screens (FIGS. 1 and 2) to view sections of the aircraft32, such as the wing54. The dashboard display28includes a toolbar70that may be customized for the particular application, and allows the user to view and call up 3-D models of the aircraft32, including parts and sections thereof, and to use the PNVS20to display an area58of the wing54containing multiple non-conformances. For example, the toolbar70may include navigation tools71, visualization tools72, tools74allowing the user to select various image viewpoints, analyzing tools76and a tool78for calling up the PNVS20. In one embodiment, coordinates65may be automatically displayed based on a point location80on the 3-D image56selected by moving a cursor (seeFIG. 4) to that location on the display screen30. Also, an electronic record45form may be displayed, showing the coordinates of the point locations of non-conformances within the area58surrounded by the displayed boundary68.

The dashboard display28may also be employed by the user to select point locations of the non-conformances on the wing54. For example, referring toFIG. 4, an electronic 3-D image56of the wing54may be called up on the dashboard display28and rotated as desired in order to show a desired section84of the wing54. The image56may be generated from the 3-D CAD files (FIGS. 1 and 2), or alternatively the image56may be generated from point cloud files50produced by scanning the aircraft32with a laser or other type of scanner62. In one embodiment, the user moves a selector such as a tablet pen or cursor82to the wing section84on the 3-D image56where a non-conformance is known to exist, based on a visual inspection of the wing. Using the cursor82, the user selects a point location80on the displayed wing section84. In response to the selection, the non-conformance capture routine42generates and enters the X,Y,Z coordinates of the selected point location80into an electronic record45(seeFIG. 8) for the particular aircraft32being mapped.

FIG. 8illustrates the fields of one typical form of an electronic record45containing the X, Y, and Z coordinates96for a group of non-conformance location points94selected by a user. The non-conformance capture routine40(FIG. 2) automatically populates the electronic record45form with the coordinates96for each of the location points94as they are selected by the user on an image56displayed on the display screen30. The electronic record45may have various other fields useful in describing areas or types of non-conformances, and associating them with other information, such as the model number88of the aircraft32, the ID # (identification number)89of the particular set of non-conformances contained in the electronic record45, a description of the non-conformance type90, comments92, and a non-conformance work order91(NCO) which may be used to associate several groups of non-conformances that may be related to each other.

Referring now toFIG. 5, upon selection of a point location80of the non-conformance on the wing section84, the point location80is displayed on the image56. Referring toFIGS. 6 and 7, the process of selecting the point locations80of non-conformances on the wing section84, and the automatic electronic entry of the X,Y,Z coordinates of these locations into the electronic record45is continued until the coordinates96(FIG. 8) of all of the non-conformances within the wing section84have been entered into the electronic record45.

Referring now toFIGS. 9 and 10, when the user has completed entering the point locations80of all of the non-conformances into the electronic record45, the non-conformance display routine42calculates three dimensional outer boundaries68of the area58containing the point locations80of the non-conformances, and displays a corresponding boundary line69in 3-D on the 3-D image56. Displaying the boundaries68in 3-D permits contours of the areas58containing non-conformances to be visualized. In the case of the example shown inFIGS. 9 and 10, it can be seen that the displayed boundaries68extend fore and aft over a curved section of the wing54and down onto a curved leading edge54aof the wing54.

FIG. 11is a software flow diagram broadly illustrating the operations performed by the non-conformance capture routine40(FIG. 2). Beginning at98, a determination is made of whether there is more than one non-conformance to be captured in a section84of the aircraft32under consideration. If the non-conformance item is not a multi-point non-conformance to be captured, i.e it is a single point, then at100a single point location including the X,Y,Z coordinates of the point are displayed by the PNVS20on the electronic 3-D display screen30. If, however, the non-conformance item comprises non-conformances at multiple location points, then the electronic record45form is displayed on the display screen30and the user may fill in the fields of an electronic record45form, identifying the model of the aircraft, a description of the non-conformance type, comments, etc.

Next, the 3-D display program46(FIGS. 1 and 2) is opened and a 3-D image of a desired section84of the aircraft32is displayed at104. At106, the user manually selects point locations on the 3-D image of the aircraft32displayed at104, and the electronic record45form is automatically populated with the X, Y, and Z coordinates of each of the selected points as the points are selected by the user. At108, the electronic record45, including the X, Y, and Z coordinates of the non-conformances, is saved by the user, e.g. by pressing a “save” button (not shown), and stored to a data repository such as an electronic memory for future recall and use in analyzing and/or correcting non-conformances. For example, at110, the electronic record45, including the non-conformance location points, may be sent to a computer numeric controlled (CNC) control room where a program controlling a CNC machine such as a robotic spray painter, is modified to reduce or eliminate future non-conformances.

FIG. 12is a software flow diagram broadly illustrating the operations performed by the non-conformance display routine42(FIGS. 1 and 2). Beginning at112, the PNVS dashboard display28is called up and connected to backend databases (“BEDB”) which may include installation plans44, 3-D CAD files52, other stored data sources48, as well as the non-conformance database26(FIG. 1) which contains the non-conformance records45. At114non-conformance records stored in a back-end database, e.g. non-conformance database26(FIG. 1), such as groups of non-conformances, may be retrieved. At116, the X,Y,Z coordinates in the coordinate fields96(FIG. 8) of the electronic records45are mapped. At118the X,Y,Z fields96are evaluated to determine whether they contain numerical coordinate data, i.e. coordinate values. If the X,Y,Z fields96do not contain numerical values, then at120, a determination is made of whether these fields contain free text that identifies the spatial locations of non-conformances. For example, in the case of the aircraft32, an X,Y,Z field96may contain a notation of “10AFT Rib” which is interpreted by the routine42to mean that non-conformances are present on the number10aft rib of the aircraft32.

If the X,Y,Z field96does not contain either coordinate values or free text that identifies spatial locations on the aircraft32, then the examination of the X,Y,Z field96ends at121. If however, the X,Y,Z fields96contain free text that define the spatial locations of non-conformances, then the backend databases are updated with the X,Y,Z coordinates of those spatial locations. Next, at124, the 3-D display is opened and the X,Y,Z coordinates of the non-conformances are retrieved, allowing the 3-D display to display a 3-D image of the aircraft32showing the locations of the non-conformances at126. Also, the backend databases are updated based on the X,Y,Z coordinates used to display the image of the aircraft32.

If the X,Y,Z field contains free text that identifies the spatial location of a non-conformance, then at122, the backend databases previously mentioned are updated with the X,Y,Z coordinate values corresponding to the spatial locations defined by the free text. Next, at124, the 3-D display is opened on the dashboard display28, and the X,Y,Z coordinates are used to display the spatial location of a non-conformance located at these coordinates. As the spatial locations of each of the non-conformances are displayed, the backend databases are updated with the X,Y,Z coordinates of the displayed non-conformance, as shown at126.

Returning now to118, if the X,Y,Z field96contains numerical data, i.e. the X,Y,Z coordinates of one or more non-conformance, then at128, a determination is made of whether the numerical data represents a single point non-conformance or multipoint (multiple) conformances. If the numerical data represents a single point non-conformance, then the location of that non-conformance is shown on the image56of the aircraft on the 3-D display at124. If, however, it is confirmed at130that the numerical data represents a multipoint non-conformance, then at132a determination is made of whether the multiple X,Y,Z coordinates are in digital numerical format. If it is confirmed that the multiple X,Y,Z coordinates are in digital numerical format, then at134the area58, including its boundaries68, are displayed on the 3-D image56, and optionally the X, Y and Z coordinates of the non-conformances within the area58are also displayed along with the image56on the display screen30. If, however, it is confirmed that the multiple X,Y,Z coordinates are not in digital numerical form at132, then the backend databases are updated with the numerical X,Y,Z coordinates of the non-conformances and they are displayed at124.

FIG. 13broadly illustrates the overall steps of a method of mapping and visualizing areas of non-conformances on an object, such as the aircraft32previously described. Beginning at138, non-conformances in the object (e.g. the aircraft32) are identified, either by visual inspection or by using automated equipment such as machine vision systems. At140, an electronic record45is generated containing the spatial locations of each of the non-conformances within a 3-D coordinate system of the object. At142, the boundaries68of an area58of the object containing the identified non-conformances are calculated based on the spatial locations contained in the electronic record45. At144, an electronic image56of the object is generated, and the boundaries68of the area58containing the non-conformances are displayed on the 3-D image56. Based on the boundaries68that are displayed, production equipment or processes may be altered to automatically correct or rework the non-conformances, or to avoid future non-conformances.

Referring now toFIGS. 14 and 15, the PNVS20may be employed to map and visualize non-conformance areas58on a group of substantially identical manufactured objects, such as a fleet of aircraft32. The PNVS20permits technicians to compare areas of non-conformances for different ones of aircraft32in a group or fleet, as well as to analyze trends in non-conformances. Based on comparison of the areas for different ones of the aircraft32in a group, adjustments may be made in one or more production processes and/or production equipment which reduce or eliminate future non-conformances, or which allow any necessary rework to be carried out more efficiently and/or effectively. Following these adjustments, the non-conformances may be reworked or corrected.

FIG. 14represents a 3-D image56of a wing section84in which the boundary lines69are shown for each of the areas58a,58b,58c,58dof non-conformances respectively for a plurality of substantially identical manufactured objects, such as a group or fleet of aircraft32. The boundary lines69of the areas58a-58dfor the different aircraft32may be superimposed on the 3-D image56of one of the aircraft32. By visualizing areas58of non-conformances for a group of objects, the areas of overlap of the boundary lines69for the areas58a-58d, indicated by the shaded area75inFIG. 14Acan be visualized. Moreover, as shown inFIG. 14A, an outer boundary line69aof the combined areas58a-58dmay be calculated and displayed on the image56to allow technicians to make appropriate changes in production processes and/or or equipment, such as, for example and without limitation, reprogramming automated equipment such as robots.

FIG. 15broadly illustrates the steps of a production method, that includes mapping and visualizing non-conformances for a group of objects, such as a fleet of aircraft as previously described in connection withFIG. 14. At146, non-conformances are identified on each of a plurality of substantially identical manufactured objects. At148, for each of the manufactured objects, an electronic record45is generated that contains the spatial locations of the non-conformances within the coordinate system of the object. At150, the non-conformances on the objects are mapped using the electronic records45. At152, outer boundaries68are calculated of an area58on each of the objects that contains non-conformances. At154, an electronic image56of the object is generated, and the boundary lines69of each of the areas58on the electronic image56containing non-conformances are displayed. The boundary lines69of the areas58may be superimposed on the electronic image56. At157, at least one process used to manufacture the objects is modified based on the boundaries68displayed on the electronic image56. Following the modification, the non-conformances within the boundaries68may be reworked or corrected.

Embodiments of the disclosure may find use in a variety of potential applications, particularly in the transportation industry, including for example, aerospace, marine, automotive applications and other application where objects, projects, or structures are manufactured, such as, without limitation, an aircraft. Thus, referring now toFIGS. 16 and 17, embodiments of the disclosure may be used in the context of an aircraft manufacturing and service method158as shown inFIG. 16and an aircraft160as shown inFIG. 17. Aircraft applications of the disclosed embodiments may include, for example, without limitation, production operations which may result in non-conformances or variations, such as, without limitation, painting an aircraft. During pre-production, exemplary method158may include specification and design162of the aircraft160and material procurement164. During production, component and subassembly manufacturing166, and system integration168of the aircraft160take place. Thereafter, the aircraft160may go through certification and delivery170in order to be placed in service172. While in service by a customer, the aircraft160is scheduled for routine maintenance and service174, which may also include modification, reconfiguration, refurbishment, and so on.

As shown inFIG. 17, the aircraft160produced by exemplary method158may include an airframe176with a plurality of systems178and an interior180. Examples of high-level systems178include one or more of a propulsion system182, an electrical system184, a hydraulic system186and an environmental system188. Any number of other systems may be included. Although an aerospace example is shown, the principles of the disclosure may be applied to other industries, such as the marine and automotive industries.

Systems and methods embodied herein may be employed during any one or more of the stages of the production and service method158. For example, components or subassemblies corresponding to production process166may be fabricated or manufactured in a manner similar to components or subassemblies produced while the aircraft160is in service. Also, one or more apparatus embodiments, method embodiments, or a combination thereof may be utilized during the production stages166and168, for example, by substantially expediting assembly of or reducing the cost of an aircraft160. Similarly, one or more of apparatus embodiments, method embodiments, or a combination thereof may be utilized while the aircraft160is in service, for example and without limitation, to maintenance and service174.