Abstract:
Systems, methods and mediums are provided for automating experiments within an automated environment without the need to disassociate the test subject (e.g., the semiconductor chip or chips) from that environment. An “experiment” may be a pre-planned deviation of an established (e.g., pre-defined) process utilizing the automated environment. 
     A computer-implemented method, system and computer-readable medium for managing experiments, such as those relating to semiconductor technology. An experiment order includes some deviation from a base process capable of operating in an automated environment. An approval of the experiment order is obtained from a distribution list of users, while permitting the users to attach documents to the experiment order or perhaps modify the experiment. The experiment order is translated into processing data suitable for implementation by said automated environment, and stored. The experiment is caused to be executed in conjunction with at least some portion of said base process via the automated environment according to the processing data.

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention concerns computer-related methods systems and mediums for managing experiments. More specifically, it relates to managing experiments concerning changes in a process, for example processes for controlling semiconductor technology manufacture. 
   2. Related Art 
   Machines, materials and processes in most industries are becoming increasingly complex and costly. Meanwhile, a need has arisen for the continuing improvement of processes and of machine and material quality. 
   Semiconductors and other products are typically manufactured under control of pre-defined processes. These pre-defined processes may be highly complex. For example, a pre-defined manufacturing process for producing semiconductor chips might contain five hundred to seven hundred and fifty steps. Moreover, each of these steps might have several variables, for example six variables, that are significant. 
   In order to improve manufacturing or test theories, it is often desirable to perform experiments by changing some small portion of the base manufacturing process. For example, an engineer might want to make one of the layers on a semiconductor ten percent thicker. This might entail performing the recipe for that step for an extra 15 seconds, with perhaps some adjustments in subsequent steps. Typically the engineer does not create a new base process including the modifications to adapt to the desired test, since that would be too time consuming. 
   Unfortunately, such an experiment using conventional techniques requires manual intervention and manual tracking of results. Accordingly, the engineer or operator performing the experiment would obtain a number of semiconductor chips and process them outside of an automated (e.g., production or mock-production manufacturing) environment. Thus, the products on which the experiment is performed need to be removed from the automated environment, which is both time-consuming and allows for the potential introduction of extraneous factors which may ultimately (and inadvertently) affect the results of the experiment. In addition, such removal of the semiconductor chips makes it difficult to coordinate manual tracking of changes or experiment history, and to control experiments and to analyze overall results. 
   Consequently, for research and development engineers, operators and other users working in factory settings, there remains a need for experiments on changes to existing processes to be flexible, easy and traceable. 
   SUMMARY OF THE INVENTION 
   The present invention alleviates the problems of the conventional techniques described above by providing systems, methods and mediums for automating experiments within an automated (e.g., production or mock-production manufacturing) environment without the need to disassociate the test subject (e.g., the semiconductor chip or chips) from that environment. An “experiment,” according to at least some embodiments of the present invention, is a pre-planned deviation of at least some portion of an established (e.g., pre-defined) process utilizing the automated environment. 
   According to at least some embodiments of the present invention, experimentation begins with an experiment order (i.e., request to initiate an experiment), which is first originated as an informal request, submitted to a computerized system, routed through various defined users, perhaps modified, and ultimately approved. In facilitating the implementation of the requested experiment, experiment management includes four conceptually distinct stages: order management, setup, execution, and analysis. The order management component of the invention assists in automatically navigating the formalization of the experiment order (mentioned above) and tracking the experiment. The setup stage typically handles the manual or automated translation of the experiment from the generalized statements, requirements, or proposed results into data defining a specific process ready to execute by the automated environment. The execution stage includes the execution of the experiment itself via the automated environment based on the process data, including the collection of experiment results. In the analysis stage, results of the experiment are reported and analyzed. 
   In accordance with at least some embodiments of the present invention, in operation, an experiment order is received, the experiment order including at least some deviation from a base process capable of operating in an automated environment. An approval of the experiment order is then obtained. At least a portion of the experiment order is translated into processing data suitable for implementation by said automated environment, and stored. The experiment is caused to be executed in conjunction with at least some portion of said base process via the automated environment according to the processing data. 
   Further, the invention may include storing data defining the experiment order, distributing the experiment order to a plurality of users, obtaining changes to the experiment order from at least one of the users, and receiving the approval for the experiment order from at least one user. Moreover, documents may be attached to the experiment request. 
   Additionally, information indicating a state change of the experiment request may be published, responsive to a document attached to the experiment request or to a change in state of the experiment order. 
   Moreover, the experiment may produce at least one test product and at least one production product (i.e., a control, which could be, e.g., a product which was processed before or after the test product, and which was processed according to the base process); the processing data may include an indication of the base process, the changes to the base process, and a split-off of a control set (i.e., the products subject to the experiment); and the split-off of a control set may produce the at least one production product according to the base process and the changes to the base process may produce the at least one test product. The results of the execution of the experiment may be stored. 

   
     BRIEF DESCRIPTION OF THE FIGURES 
     The above mentioned and other advantages and features of the present invention will become more readily apparent from the following detailed description in the accompanying drawings, in which: 
       FIG. 1  is a block diagram of a computerized process control system which may be used in connection with at least some embodiments of the present invention. 
       FIG. 2  is a flow chart of an overall process for experiment management according to at least some embodiments of the invention. 
       FIGS. 3A and B  are a flow chart of an order management process portion of the overall process of FIG.  2 . 
       FIG. 4  is a flow chart of a setup process portion of the overall process of FIG.  2 . 
       FIG. 5  is a flow chart of an execution process portion of the overall process of FIG.  2 . 
       FIG. 6  is a flow chart of an analysis process portion of the overall process of FIG.  2 . 
       FIG. 7  is a diagram illustrating definition of an experiment. 
       FIG. 8  is an exemplary user interface for an experiment editor, used in connection with at least some embodiments of the present invention. 
       FIG. 9  is an exemplary user interface for the experiment editor, illustrating attachments, used in connection with the invention. 
       FIG. 10  is an exemplary user interface for an experiment editor, illustrating experiment content, used in connection with at least some embodiments of the present invention. 
       FIG. 11  is an exemplary user interface for an experiment editor, illustrating wafer level split details, used in connection with at least some embodiments of the present invention. 
       FIG. 12  is an illustration of at least some embodiments of an experiment. 
   

   DETAILED DESCRIPTION 
   The following detailed description includes many specific details. The inclusion of such details is for the purpose of illustration only and should not be understood to limit the invention. Throughout this discussion, similar elements are referred to by similar numbers in the various figures for ease of reference. 
   As indicated above in the Summary section, an “experiment,” according to at least some embodiments of the present invention, is a pre-planned deviation of at least some portion of a base process utilizing an automated environment. Typically an experiment is performed on materials, such as semiconductor chips, that are produced as a result of the automated process. Also as indicated above, at least some embodiments of the present invention envision that experiment management includes four conceptually distinct stages: order management, setup, execution, and analysis. Although these stages are conceptually distinct, they may temporally overlap. 
   According to at least some embodiments of the present invention, reports, memos, forms, files, and other documents may be associated with a particular experiment throughout the order management and setup stages. These may be reviewed by users allowed access to the experiment. This permits users and reviewers to comment on the experiment, provide background information, provide appropriate forms, attach relevant information, etc., in a user-friendly, highly flexible fashion. Due to its flexibility, it invites users to provide input and should result in higher quality experiments. 
   Reference is now made to  FIG. 1 , a block diagram generally illustrating a computerized process control system which may be used in connection with at least some embodiments of the present invention. As is illustrated, the experiment order  101  is input to a computerized system, referred to generally as a controller  103 . The experiment order  101  contains a description, such as in text, of a desired experiment. The experiment order  101  could be, for example, a word processing document containing text. As one alternative, it could be input from a menu. The experiment described in the experiment order  101  is a deviation from an existing automated process for creating a product, although it is not necessarily described in the order as a deviation from a particular process. 
   The controller  103  has access to various stored processes  111 , such as manufacturing processes for semiconductor chips. The controller  103  could be a general purpose computer, or a special purpose computer specially programmed, or other automated system or distributed system. (In general, such computers as used here, or whose use may be apparent from the context of the discussion, can be any number of different types of computers, including those containing processors from Intel Corporation of Santa Clara, Calif., wherein these computers can contain any number and different types of storage devices serving as computer-readable mediums; in addition, it is contemplated by at least some embodiments of the present invention that the computer-readable medium be a transmission). The stored processes  111  comprise a number of automated steps in a manufacturing process. The actual format of the contents of these steps is defined by the system and devices in the system. Some of the steps in the processes utilize recipes, stored in a recipe database  113 . Recipes may be shared by various processes. The controller  103  controls the processing of an automated environment such as production system  105 , which ultimately produces production products  107 , or following an experiment, produces test products  109 . The invention thereby allows users to submit experiment requests, create derivations of base processes, and to track the status of experiment requests. 
   Reference is made to  FIG. 2 , a flow chart of an overall process for experiment management according to at least some embodiments of the present invention. The four conceptual stages (as mentioned above) included: order management  201 , Manufacturing Execution System (MES) setup  203 , execution  205 , and analysis  207 . 
   At the order management stage  201 , further defined below, the experiment order is defined. Typically, an experiment would be defined in the experiment order as a set of requirements, and may be specified as a deviation from an existing process. The experiment order is subject to routing, review, and change by various personnel, prior to being approved for the next stage. 
   At the MES setup stage  203 , the experiment order is translated into the experiment setup, that is, specific processing data which can be executed by components in the production system. The processing data is in a format which is expected by the production system components. In typical situations, data to execute the experiment is interjected between (and/or replaces existing) steps of a base process. 
   At the execution stage  205 , the execution of materials is performed, based on the experiment setup. Most or all of this stage is performed automatically by the production system components. The results of each step in the setup implemented at this execution stage  203  are recorded. 
   At the analysis stage  207 , the results of the experiment are reported and analyzed. This may be done automatically by a computer, and/or may include analysis by the user. 
   Reference is made to  FIGS. 3A and 3B , a flow chart of an example order management stage  201  of the overall process of  FIG. 2 , as envisioned by at least some embodiments of the present invention. This stage allows the experiment to be requested and be performed following experiment request review and sign-off. At step  301 , the experiment is initially defined by a requestor. In order to facilitate experiments, it is envisioned that requests can be submitted in any appropriate form. One appropriate form is a textual description in an electronic document. Note that the experiment may be informally described. It is not necessary for the initial experiment request to define the experiment as a variation from an existing process. 
   At step  303 , the experiment object (or other storage for experiment data) is created. Initial information is collected to identify the requestor and the experiment. The information is stored, such as in an object. The experiment request is then distributed to appropriate users identified in a distribution list. 
   At step  305 , a user who received the experiment request (e.g., for review) may attach external files, memos, forms, or other documents to the experiment request. The ability to associate documents with the experiment request can be used to facilitate user interaction concerning the experiment request. These documents may then be reviewed by other users. At step  307 , the user (or automated entity) determines the changes to be made to a particular base process. The user (or automated entity) may also determine the base process which is to be modified. Also, at step  309 , the user (or automated entity) will determine when to split off a lot from the control set, and the lot-specific transactions that are to be made. At step  311 , the user (or automated entity) determines what recipe changes, if any, need to be made. Having determined the specified changes to be made to the base process, the system receives and stores the changes as processing data. At step  313 , the experiment, as it has been tweaked by the users, is sent for sign-off, described in FIG.  3 B. At step  315 , if the experiment has been approved by the users, the process ends  317  and the experiment proceeds to the next conceptual stage. Otherwise, the process returns to step  305  for further handling. 
     FIG. 3B  illustrates one embodiment of the sign-off process. At step  321 , a user who received the experiment request (e.g., for review) may attach external files, memos, forms, or other documents to the experiment request, which may then be reviewed by other users. At step  323 , if documents are attached or deleted to the experiment request, or at step  325  if there was a state change for the experiment request, such information is published  327 . One appropriate method for publication is to send such information to listed users via e-mail. A state change would include, for example, a “sign-off” on the experiment (or portion thereof). At step  329 , if an indication of final approval (or affirmative lack of approval) has not been received, the process repeats at step  321 . If final approval has been received, the stage is ended  331 . 
   Reference is made to  FIG. 4 , a flow chart of a setup stage  203  portion of the overall process of FIG.  2 . During the setup stage, a user can set up the particular experiment. For example, a user could set up experiment-specific data, for example a reticle or recipe details. At step  401 , a user (or automated entity) retrieves and reviews the experiment order. As indicated above, the experiment order may be an informal description of an experiment. A user can determine how a process should be implemented to effect the requested experiment, or the process can be automated, for example, by parsing the description of the experiment and identifying certain key words or phrases that are indicative of what is requested. At least some embodiments of the present invention envision that this can be done utilizing, e.g., various expert system techniques. At least some embodiments of the present invention also envision some combination of automation and user participation. 
   Still referring to  FIG. 4 , at step  403 , the user (or automated entity) determines the changes to be made to a particular base process. The user (or automated entity) may also determine the base process which is to be modified. Also, at step  405 , the user (or automated entity) will determine when to split off a lot from the control set, and the lot-specific transactions that are to be made. At step  407 , the user (or automated entity) determines what recipe changes, if any, need to be made. Having determined the specified changes to be made to the base process, the system receives and stores the changes as processing data. 
   Reference is made to  FIG. 5 , a flow chart of an execution stage  205  of the overall process of FIG.  2 . At this point, the experiment has been defined in processing data which can be input to the automated environment. The experiment can then be processed in a manner which is transparent to the automated environment. At step  501 , the automated environment receives the processing data for the modified process. At step  503 , the automated environment executes a step of the processing data. If there are any test results to be stored, at steps  505 - 507 , the system stores the test results. At step  509 , if processing is not complete, the automated environment returns to continue processing at step  503 . When processing is complete, this stage ends at step  511 . 
   Reference is made to  FIG. 6 , a flow chart of an analysis stage  207  of the overall process of FIG.  2 . Experiment history setup information and history data is available for use in analysis and reporting. The experiment results are collected at step  601 . At step  603 , the experiment results are made available for any analysis. For example, a user may wish to make a manual analysis of the results. At step  605 , the automated environment performs any requested computerized analysis. If there are any proposed changes to the experiment, at steps  607 - 609 , the user may generate another experiment request. The analysis is completed at step  611 . 
   Reference is made to  FIG. 7 , a diagram illustrating the defining of an experiment, as contemplated by at least some embodiments of the present invention. Specifically, the experiment  701  initially is associated with stored data including attribute information  703 , for example defined by the user, and operation information  705 , defining how the experiment operates. An experiment initially may be created from scratch, or may be copied from another experiment used as a template. Typical attributes would include sufficient information to identify useful information about the experiment, such as an experiment identifier, an experiment objective, a requestor name, an experiment name, a requestor e-mail address. 
   When the experiment is initially defined, a starting state will be “underchange”  707  (indicating that the experiment may be changed), and once the experiment is approved, the ending stage is effective (distributed)  711 . There may be a series of user-defined states  709  which are under control of the user, subsequent to the underchange state, and prior to the effective state. The effective state is entered after the experiment is approved and signoff is obtained. Preferably, a user cannot change the contents of an experiment without appropriate permission. There may be other user-defined attributes, as well as attached external documents and/or files, and a user-defined state model. According to one possible implementation, the experiment is implemented as an object. Note that this state table corresponds to the order management process portion. 
     FIGS. 8-11  are examples of a potential user interface to be used in connection with at least some embodiments of the present invention. First, reference is made to  FIG. 8 , one aspect of an exemplary user interface for an experiment editor. Here, the user may provide information about the experiment  811 , about experiment attributes  813 , and optionally about experiment category  815 . Experiment information may include an objective  801 , which may summarize a description of the experiment. Other experiment information includes requestor identification information  803  (for example, name, e-mail address); the basic process or state model  805  for the experiment; and optionally an effective date  807  after which the experiment request will expire. The information collected in this initial interface is associated with the experiment request. 
   Reference is made to  FIG. 9 , another aspect of an exemplary user interface for the experiment editor, illustrating attachments used in connection with at least some embodiments of the present invention. In such embodiments, documents such as files, memos, forms, web addresses, etc., without limitation, may be attached to or otherwise associated with the experiment request.  FIG. 9  lists, by way of example, several documents, by file name  909 , which are attached to the experiment request: a local document experiment doc  901 ; a filepath for another document C:\Experiment\Experiment.doc  903 ; a web site www.consolium.com  905 ; and an http document http://www.consilium.com/corp_events.html?phase=ge  907 . The user interface of the present example also indicates whether or not the file is simply a reference  911 . 
   Reference is made to  FIG. 10 , another aspect of an exemplary user interface for the experiment editor, illustrating experiment content, used in connection with at least some embodiments of the present invention. This exemplary user interface allows access to experiment content  1001 , physical split details  1003 , and merge details  1005 , the split treating the standard and test materials differently, and the merge detailing how the standard and test materials are treated when merged after the split. The experiment content  1001  provides the file controlling the experiment process. Here, it names the experiment process  1007 , the experiment route  1009 , and the experiment operation  1011 . Note that additional information on the experiment may be provided, such as whether the processing is pre- or post-split  1013 . 
   Reference is made to  FIG. 11 , an exemplary user interface for the experiment editor, illustrating wafer level split details, used in connection with the invention. Here, the processing data provides specifics, at lot level, slot level, or unit level  1101 . The present example concerns a slot level split. As is illustrated, the split details  1103  provide the slots and the quantity to be split; as well as the process plans  1105  to be associated with each split. 
   Reference is made to  FIG. 12 , illustrating at least some embodiments of an experiment as contemplated by the present invention. Each experiment order  1201  may have associated with it various documents, such as files  1203 , forms  1205 , memos  1207 , and experiment results  1209 . Users can add or delete the document to/from the experiment order. Preferably, an attachment of a document will be considered an event, and may result in the publication of the event for example by e-mail or Workflows. 
   An experiment order may be copied by a user, together with attached documents, attributes, and other correlated information 
   Also, according to at least some embodiments of the present invention, changes to the experiment order are stored in a history. Stored changes could include changes to native attributes, external document additions/deletions, and associated with other objects. 
   Consider an example of an experiment, with reference to  FIGS. 3 through 6 . In this example, the user wants a specified layer of a chip to be 10% thicker. The experiment in this example is an idea from an engineer. The experiment request is defined by a user, and submitted to the system at step  301  through  303 . It could be a very general request with a simple textual description. An experiment object is created for the experiment request, and the experiment request is routed to the appropriate users for approval, at steps  305  through  313 . The approval may be automated, such as delivery via e-mail awaiting a marking as approved. As shown in steps  321  through  329 , until sign-off is received for the experiment, users may attach and/or delete relevant files, memos, etc. to the experiment object. If there are attachments or deletions, or if the experiment has changed state, the event is published to the users, shown in steps  323  through  327 . The review process continues until sign-off is received. 
   Once sign-off is received, the experiment order is reviewed and translated to processing data, as shown in FIG.  4 . This review and translation may be a manual process done by a person with the appropriate experience. In addition, it may also be performed (in whole or part) by automated means. In any event, it could be determined at step  403 , for example, that wafers  1 - 11  in the lot will be the control (i.e., the established steps will not be effected), and the remainder of the wafers in the lot will be the test product. Also, it could be determined that a particular parameter in the 500 th  cycle of a standard base process must be changed from 100 to 200. It would be specified at step  405  that the controls will be split off from the other processing. If it was necessary, a new recipe would be created or an existing recipe would be modified at step  407 . All of the wafers will be under automated control. The two lots will be re-united and held or delivered for analysis. The information related to the variations from the base process, specific execution transactions, and any recipe change are stored as processing data. Note that the experiment could call for additional or different information to be collected as part of the processing results. 
   The experiment is then run, as shown in FIG.  5 . At this point, the experiment processing data are handled no differently from a regular control job. That is, no exception processing is required. The processing data is input into the manufacturing system at step  501 , and the test proceeds automatically by executing the processing data at step  503 . Test results that are generated during execution of the experiment are stored at steps  505 - 507 . 
   Following the experiment, test product might be reclassified from test materials to standard production materials, if within tolerances, and shipped to customers. Alternatively, the non-standard processed materials could be scrapped, or saved for further analysis, as shown in FIG.  6 . 
   While this invention has been described in conjunction with the specific embodiments outlined above, many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the preferred embodiments of the invention as set forth are intended to be illustrative and not limiting. Various changes may be made without departing from the spirit and scope of the invention as defined in the following claims. 
   For example, it would be possible to define an entire experiment from scratch. A typical semiconductor manufacturing process is 500 to 750 steps, so it may often be more efficient to define an experiment as a variation from an existing process. 
   As another example, the controller may be a general purpose computer, a specially programmed special purpose computer; it may also be implemented as a distributed computer system rather than as a single computer.