Patent Publication Number: US-2006004859-A1

Title: Methods and apparatus that use contextual test number factors to assign test numbers

Description:
BACKGROUND OF THE INVENTION  
      Some forms of circuit test provide an engineer with large volumes of test results. To assist the engineer in managing and referring to these test results, each test result may be associated with a test number. Typically, the assignment of test numbers begins with some arbitrary base number (e.g., 1 or 1000). A new test number is then generated by merely incrementing the base, or last-assigned, test number.  
     SUMMARY OF THE INVENTION  
      Several aspects of the invention are embodied in methods for assigning test numbers. Common to each method is the maintenance of current testflow context information during execution of a testflow.  
      A first of the methods proceeds as follows. If one or more test number factors have been specified for one or more levels of the current testflow context, the test number factors are used to determine a base number for a current test number range. Otherwise the base number is set to a default base number. Upon execution of a subtest in the testflow, the result of the subtest is then assigned a next test number in the current test number range.  
      In a second of the methods, upon execution of a subtest, a database of test numbers is indexed using index information comprising 1) an identifier of the subtest, and 2) the current testflow context information. If a test number corresponding to the index information exists in the database, the test number is assigned to a result of the subtest. If a test number corresponding to the index information does not exist in the database, a new test number is assigned to the result, and the index information and new test number are associated in the database. The new test number is assigned as follows. If one or more test number factors have been specified for a level of the current testflow context, the test number factors are used to determine a base number for a current test number range. Otherwise the base number is set to a default base number. The new test number is then set to a next test number in the current test number range.  
      In a third of the methods, upon execution of a subtest in a testflow, a database of test numbers is indexed using index information comprising i) an identifier of the subtest, and ii) the current testflow context information. If a test number corresponding to the index information exists in the database, the test number is assigned to a result of the subtest. If a test number corresponding to the index information does not exist in the database, a new test number is formed by applying an increment to a previously assigned test number (with said increment corresponding to a level of the current testflow context). The new test number is then assigned to the result of the subtest, and the index information and new test number are associated in the database.  
      Another aspect of the invention is embodied in a test number engine. The test number engine comprises computer readable media, and program code that is stored on the computer readable media. The program code comprises code to, if one or more test number factors have been specified for one or more levels of the current testflow context, use the test number factors to determine a base number for a current test number range, or otherwise set the base number to a default base number. The program code further comprises code to, upon execution of a subtest in the testflow, assign a result of the subtest a next test number in the current test number range.  
      Other embodiments of the invention are also disclosed.  
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
      Illustrative and presently preferred embodiments of the invention are illustrated in the drawings, in which:  
       FIG. 1  illustrates an exemplary method for assigning test numbers;  
       FIG. 2  illustrates an exemplary testflow;  
       FIG. 3  illustrates a progression of testflow context information for the  FIG. 2  testflow;  
       FIG. 4  illustrates an exemplary database of test number ranges;  
       FIG. 5  illustrates an exemplary portion of a test number database that may formed during execution of the  FIG. 2  testflow;  
       FIG. 6  illustrates a second exemplary method for assigning test numbers;  
       FIG. 7  illustrates a database of test numbers that may be generated in response to execution of the  FIG. 2  testflow;  
       FIG. 8  illustrates a database of test results that may be generated in response to execution of the  FIG. 2  testflow;  
       FIG. 9  illustrates a third exemplary method for assigning test numbers; and  
       FIG. 10  illustrates an exemplary test number engine.  
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT  
       FIG. 1  illustrates a first exemplary method  100  for assigning test numbers. In accordance with the method  100 , current testflow context information is maintained  102  during execution of a testflow.  
      As defined herein, a “testflow” is any portion of a test program that is used to specify the type, number or order of tests that may be executed during circuit test. Testflow context information may comprise any information that assists in defining what portion of a test program is being executed. By way of example, testflow context information may comprise information such as a test suite designator, a port designator, a vector label, a vector designator, or a pin designator. Testflow context information may also comprise a loop designator and/or loop iteration designator for each of a number of loop layers (e.g., nested loops) that have been entered during execution of a testflow. As used herein, the term “designator” encompasses a string, number or any other means that can be used to designate a test suite, loop or other testflow context.  
      If one or more test number factors have been specified for one or more levels of the current testflow context, the test number factors are used  104  by the method  100  to determine a base number (e.g., a beginning number or reference number) for a current test number range. Otherwise, the base number is set to a default base number.  
      In one embodiment of the method  100 , the base number is compiled by aggregating (e.g., summing or multiplying) a plurality of test number factors that have been specified for various levels of the current testflow context. By way of example, the test number factors may comprise increments or addends that are specified at any or all of: a loop level, a test suite level, or a subtest level.  
      In another embodiment of the method  100 , a user may only be allowed to specify a single base number per testflow context. In this embodiment, the base number for the current test number range is determined by merely setting the base number equal to the user-specified base number. In this method, there is no aggregation of test number factors. However, a user may still provide base numbers at various points in a testflow, thereby causing certain test numbers to be grouped, for example. Further, because the base numbers are set contextually, test numbers that are assigned for a later test suite do not need to continue from the test numbers assigned for an earlier test suite. Further, if a base number is specified for one test suite, it is not automatically preserved for use with the next test suite.  
      Upon execution of a subtest in the testflow, the result of the subtest is then assigned  106  a next test number in the current test number range. A “subtest”, as defined herein, may be a test that produces multiple test results, but is preferably a test or portion of a test that produces only a single test result.  
       FIG. 2  illustrates an exemplary testflow  200  for which testflow context information may be maintained. By way of example, the testflow  200  could be a system-on-a-chip testflow of the Agilent 93000 SOC Series tester (manufactured by Agilent Technologies, Inc. of Palo Alto, Calif., USA). The testflow  200  comprises five test suites  202 ,  204 ,  206 ,  208 ,  210 , respectively named AAA, BBB, CCC, DDD and EEE. Each of the test suites  202 - 210  serves to contain and/or specify a number of subtests, and may include one or more test controls (e.g., test methods, test functions or user procedures). The testflow  200  also comprises two loops  212 ,  214 , respectively named Loop_X and Loop_Y. By way of example, each loop  212 ,  214  specifies that its loop is to be iterated three times (i.e., 1..3).  
       FIG. 3  illustrates a progression of testflow context information  300  that may be maintained by the method  100  during execution of the testflow  200 . Note that the initial testflow context  302  is merely “AAA”, or the name of the first encountered test suite. After the test suite AAA has been executed, the testflow context then changes to “L1”, which is indicative of the first iteration of Loop_X.  
      Upon entry into the test suite BBB, the testflow context then changes to “BBB:L1”. Note that, for ease of reading, the testflow contexts shown in  FIG. 3  adopt the convention of always placing the current test suite&#39;s name at the beginning of the context. However, this is certainly not required by the method  100 , and need not be done.  
      Upon first entering Loop_Y, the testflow context changes to “L1:L1”. The identities of Loop_X and Loop_Y are therefore inferred from the number and order of loop iterations maintained in the testflow context. However, the identities of the loops could also be called out with specificity, such as by noting the context as “X1:Y1”.  
      Various contexts of the testflow  200  are associated with test number factors. That is, Loop_X is associated with the addend 10,000, test suite BBB is associated with the addend 200, test suite CCC is associated with the addend 220, Loop_Y is associated with the addend 2000, and test suite DDD is associated with the increment 5. Adjacent each testflow context,  FIG. 3  illustrates the aggregate base number for the context. Thus, the base number for test suite BBB is 10,200 (10,000+200); the base number for test suite CCC during a first iteration of Loop_X is 10,220 (10,000+220); the base number for test suite DDD during a first iteration of Loop_X and first iteration of Loop_Y is 12,000 (10,000+2,000); and the base number for test suite EEE is 1200. However, the base number for test suite AAA is 1, which is the default base number.  
      If a test number factor is provided at a loop level (e.g., the addend 10,000 associated with Loop_X), the loop&#39;s test number factor may be factored into a base number repeated times. For example, consider the execution of test suite DDD during the second iteration of Loop_X and third iteration of Loop_Y. The base number for this context can be calculated as (10,000*2)+(2,000*3), or 26,000.  
      Assume now that each of the test suites  202 - 210  shown in  FIG. 2  comprises three subtests, identified as Subtest1, Subtest2, and Subtest3. Note that although subtests of the same name may appear in each of the test suites  202 - 210 , they need not be (and likely will not be) the same subtest. During execution of the testflow  200 , the test numbers assigned to test suite AAA would be 1, 2 and 3. The test numbers assigned to test suite BBB during a first iteration of Loop_X would be 10,200, 10,201 and 10,202. Similarly, the test numbers assigned to test suite CCC during a first iteration of Loop_X would be 10,220, 10,221 and 10,222. The test numbers assigned to test suite DDD during first iterations of Loop_X and Loop_Y would be 12,000, 12,005 and 12,010 (i.e., numbers incremented by 5). Finally, the test numbers assigned to test suite EEE would include the integers 1200, 1201 and 1202.  
      Note that, for some testflows, different contexts of the testflow might intentionally (or unintentionally) share the same test number range. However, test numbers assigned to the two test suites will not collide, because each subtest draws a “next number” from the range, regardless of whether other test suites have already drawn numbers from the range.  
      Also note that two or more test suites may not only have the same base number, but may be associated with test number ranges that overlap or are interleaved. For example, if two test suites were respectively associated with base numbers of 1200 and 1201, and an increment of five, one test suite might be associated with the test numbers 1200,1205 and 1210, while the other test suite might be associated with the test numbers 1201, 1206 and 1211. An overlap of test numbers might similarly result from test suites that are respectively associated with the base numbers 1200 and 1301, and an increment of two.  
      If the base number that is compiled for the current test number range is new, the base number may be used to initialize a new test number range in a database  400  of test number ranges. Then, for each test number range in the database  400 , the test numbers that have been assigned from the test number range may be tracked. One way to do this is to store each range  402  along with the maximum test number  404  that has been assigned from the range. Then, each time a new test number is assigned from a range, its previous maximum test number may be incremented. Depending on the test factors that a user has assigned to a testflow, the increment may be a default increment, or an increment that a user has specified for a particular level or levels of the testflow (e.g., an increment that is associated with a level of a current testflow context).  
      As test numbers are assigned to results, a test number database may be compiled by storing each test number along with the base number from which it was derived. Optionally, the test number database may be integrated with a results database, such that a result, its test number, and the base number from which the test number was derived, are all stored in the same database. A portion of such a database  500  is shown in  FIG. 5 . Alternately, the test number database may be integrated with testflow context information, such that a test number, its testflow context information, and the base number from which the test number was derived, are all stored in the same database. This latter form of database is illustrated in  FIG. 7 .  
      After storing the database  500  or  700 , but prior to a subsequent testflow execution, the database  500  or  700  may be read to identify the various base numbers stored therein. For each identified base number, a test number range may be initialized, and the database  500  or  700  may be parsed to determine the extent of test numbers that have already been derived from the test number range. Then, during a subsequent testflow execution, the initialized test number ranges may be used to assign any new test numbers corresponding to the initialized test number ranges.  
      In one embodiment, the method  100  may be combined with the method for assigning test numbers disclosed in the United States patent application of Robert S. Kolman, et al. entitled “Method and Apparatus for Assigning Test Numbers” (Atty. Dckt. No. 10040433-1, filed the same day as this application). In accordance with this combination,  FIG. 6  illustrates a second exemplary method  600  for assigning test numbers. In method  600 , current testflow context information is maintained  602  during the execution of a testflow. Upon execution of a subtest in the testflow, a database of test numbers is indexed  604  using index information comprising 1) an identifier of the subtest, and 2) the current testflow context information. If a test number corresponding to the index information exists in the database, the test number is assigned  606  to a result of the subtest. If a test number corresponding to the index information does not exist in the database, a new test number is generated  608 . The new test number is generated as follows. If one or more test number factors have been specified for one or more levels of the current testflow context, the test number factors are used  610  to determine a base number for a current test number range. Otherwise, the base number is set to a default base number. The new test number is then set  612  to a next test number in the current test number range.  
      Turning again to the exemplary testflow  200 , upon execution of the first subtest in the testflow  200 , index information comprising an identifier of the subtest (Subtest1) and the current testflow context information (AAA) is used to index a database of test numbers. If this is the first execution of the testflow  200 , the database will be empty, and a new test number (e.g., 1) will be assigned to the result of Subtest1. The new test number (1) and index information (AAA:Subtest1) will also be associated in the database. During the first execution of the testflow  200 , these steps continue, with the use of each successive index resulting in a database “miss”, thereby causing a new test number, its associated index information (and possibly the base number from which it was derived) to be added to the database. Thus, after a first execution of the testflow  200 , the database of test numbers  700  shown in  FIG. 7  may have been created. At the same time, each newly created test number will be assigned to a test result of its corresponding subtest, thereby resulting in the database of test results  800  shown in  FIG. 8 . Although the test results shown in  FIG. 8  are all presented in terms of “pass” or “fail”, the test results of an actual test run could also or alternately comprise voltage readings, current readings, impedance measurements, and other sorts of test results.  
      Preferably, each of the subtests within a test suite is provided a unique subtest name, and enough testflow context information is maintained to ensure that each index into the database of test numbers  700  forms a unique subtest identifier. It is also preferred that each new test number entered into the database  700  is unique from all other test numbers in the database. However, the method  400  can often provide useful test numbers even when the above controls are not maintained.  
       FIG. 9  illustrates a third exemplary method  900  for assigning test numbers. In accordance with the method  900 , current testflow context information is maintained  902  during the execution of a testflow. Upon execution of a subtest in a testflow, a database of test numbers is indexed  904  using index information comprising i) an identifier of the subtest, and ii) the current testflow context information. If a test number corresponding to the index information exists in the database, the test number is assigned  906  to a result of the subtest. If a test number corresponding to the index information does not exist in the database, a new test number is formed  908  by applying an increment to a previously assigned test number, with the increment corresponding to a level of the current testflow context. The new test number is then assigned  910  to the result of the subtest, and the index information and new test number are associated in the database. In this embodiment of the invention, a user is able to assign test number increments within the various contexts of a testflow.  
      Any of the above methods may be implemented using program code. By way of example,  FIG. 10  illustrates a test number engine  1000  that can be used to implement the method  100 . The test number engine  1000  is embodied in program code stored on computer readable media (e.g., a magnetic or optical disk, a fixed or removable disk, or random access or read-only memory (RAM or ROM)). In some embodiments, the program code of the test number engine  1100  may be distributed among various computer readable media associated with one or a plurality of computer systems.  
      As shown, the test number engine  1000  may comprise code  1002  to, if one or more test number factors have been specified for one or more levels of the current testflow context, use the test number factors to determine a base number for a current test number range, or otherwise set the base number to a default base number. In one embodiment, the code  1002  does this by aggregating a plurality of test number factors that have been specified for various levels of a current testflow context. The program code also comprises code  1004  to, upon execution of a subtest in the testflow, assign a result of the subtest a next test number in the current test number range. In one embodiment of the test number engine  1000 , the code  1006  increments a previous maximum test number that has been assigned from the current test number range, by an increment specified for a level of the current testflow context.  
      While illustrative and presently preferred embodiments of the invention have been described in detail herein, it is to be understood that the inventive concepts may be otherwise variously embodied and employed, and that the appended claims are intended to be construed to include such variations, except as limited by the prior art.