Source: https://patents.google.com/patent/US20020078401A1/en
Timestamp: 2018-09-26 06:00:20
Document Index: 737969394

Matched Legal Cases: ['arts 218', 'arts 220', 'arts 228', 'arts 230', 'arts 238', 'arts 240', 'art 356', 'art 358', 'art 366', 'art 368']

US20020078401A1 - Test coverage analysis system - Google Patents
Test coverage analysis system Download PDF
US20020078401A1
US20020078401A1 US09738068 US73806800A US2002078401A1 US 20020078401 A1 US20020078401 A1 US 20020078401A1 US 09738068 US09738068 US 09738068 US 73806800 A US73806800 A US 73806800A US 2002078401 A1 US2002078401 A1 US 2002078401A1
US09738068
MICRO-ASI Inc
The present invention provides a system, apparatus and method for determining a test coverage for a device by receiving test data from a test platform or development tool, processing the test data to determine or predict the test coverage for the device, and storing the test data and test coverage in a standard format. The test data includes the results of two or more tests on the device.
The present invention relates generally to the field of test engineering and, more particularly, to a test coverage analysis system. [0001]
One of the more difficult tasks to perform in Test Engineering is to determine or predict the test coverage of a device under test (“DUT”) and unit under test (“UUT”) on an in-circuit test (“ICT”) platform or development tool. Currently, the process to extract the test data from the test platform or development tool and manipulate that data into a coverage report can take several days. Because this process is performed manually, there are also many opportunities for mistakes. Development tools that are now available do not have any built-in predicted coverage analysis tools. Also, the in-circuit testers' test analysis tools can make incorrect assumptions on coverage, resulting in inaccurate test coverage reports. Accordingly, there is a need to for a test coverage analysis system that overcomes these problems. [0002]
The present invention automates the process of generating test coverage reports. By minimizing the human intervention, the present invention minimizes the time required to generate a test coverage report and minimizes the errors in the generated reports. This allows feedback to improve in-circuit test coverage to be obtained and utilized in a more efficient and timely manner. [0003]
More specifically, the present invention provides a method for determining a test coverage for a device by receiving test data from a test platform or development tool, processing the test data to determine or predict the test coverage for the device, and storing the test data and test coverage in a standard format. The test data includes the results of two or more tests on the device. [0004]
In addition, the present invention provides a method of measuring the actual and predicted results against each other to determine the accuracy of the predicted test coverage. This method uses the standard format test coverage data that was stored for the DUT. [0005]
The present invention also provides an apparatus for determining a test coverage for a device that includes an interface, a processor and a memory. The interface extracts test data from a test platform or development tool. The test data includes the two or more tests on the device. The processor is communicably linked to the interface and processes the test data to determine or predict the test coverage for the device. The memory is communicably linked to the processor and stores the test data and test coverage in a standard format. [0006]
In addition, the present invention provides a system for determining a test coverage for a device that includes a test platform or development tool, an interface, and a memory. The test platform or development tool generates test data, which includes the results of two or more tests on the device. The interface extracts test data from a test platform or development tool. The test data includes the two or more tests on the device. The processor is communicably linked to the interface and processes the test data to determine or predict the test coverage for the device. The memory is communicably linked to the processor and stores the test data and test coverage in a standard format. [0007]
The present invention also provides a computer program embodied on a computer readable medium for determining a test coverage for a device. The computer program includes a code segment for receiving test data from a test platform or development tool, a code segment for processing the test data to determine or predict the test coverage for the device, and a code segment for storing the test data and test coverage in a standard format. The test data includes results of two or more tests on the device.[0008]
Other features and advantages of the present invention will be apparent to those of ordinary skill in the art upon reference to the following detailed description taken in conjunction with the accompanying drawings. [0009]
The above and further advantages of the invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which: [0010]
FIG. 1 is a block diagram of a preferred embodiment of the present invention; [0011]
FIG. 2 is an illustration of a first embodiment of a standard format of the present invention; and [0012]
FIG. 3 is an illustration of a second embodiment of a standard format of the present invention.[0013]
While the making and using of various embodiments of the present invention are discussed in detail below, it should be appreciated that the present invention provides many applicable inventive concepts which can be embodied in a wide variety of specific contexts. The specific embodiments discussed herein are merely illustrative of specific ways to make and use the invention and do not delimit the scope of the invention. The discussion herein relates to test engineering, specifically to automating the processing of test data to produce test coverage reports. [0014]
The present invention automates the process of generating test coverage reports. By minimizing the human intervention, the present invention minimizes the time required to generate a test coverage report and minimizes the errors in the generated reports. This allows feedback to improve in-circuit test coverage to be obtained and utilized in a more efficient and timely manner. [0015]
The present invention extracts test data from a test platform or development tool, such as Agilent 3173 or FabMaster, and processes that data into a standard presentation format. The processing may involve storing the raw test data in a computer readable structure such as a database. The standard presentation format can be, for example, a spreadsheet. [0016]
FIG. 1 is a block diagram of a preferred embodiment of the present invention [0017] 100. Platform 110 contains the test data. Possible inputs 112, 114, 116, 118, 120 and 122 indicate representative platforms from which test data can be obtained. These inputs 112, 114, 116, 118, 120 and 122 can be from in-circuit testers such as Agilent 116 or from development tools such as FabMaster 122 or from a platform not shown herein. Inputs 112, 114, 116, 118, 120 and 122 are intended solely as examples and should not be taken to limit the possible inputs to those shown in block 110, FIG. 1. Each type of platform 110 has a standard ASCII file or report that can be exported from its proprietary database. Input processor 130 exports the data from platform 110. This export can be accomplished by accessing the standard ASCII files or reports. Each platform 110 can require a customized input processor 130 in order to access the data. Input processor 130 then stores the exported test data in test data database 140. Test data database 140 can be made by any database tool or platform. Alternative computer readable storage structures, such as spreadsheets and text files, can be used instead of a database. Output processor 150 exports the data from the storage structure, test data database 140, into a standard presentation format, such as a standard test coverage report 160. Alternative standard presentation formats, such as spreadsheets and text files, can be used.
FIGS. 2 and 3 illustrate embodiments of a standard format of the present invention. Both are displayed in spreadsheet format although either may be presented in an alternative form, such as a text file. FIG. 2 illustrates a sample summary test coverage report that displays the results of calculations performed on the detailed data. FIG. 3 illustrates a sample detailed test coverage report that lists each part tested and specific test data for each. Color coding can also be integrated into the standard presentation format to draw immediate attention to areas that do not attain an acceptable rating. [0018]
More specifically, FIG. 2 includes the project name [0019] 202, CCA# 204, Date 206, Test Engineer Name 208, CCA Name 210 and ICT Testability Effectiveness (Predicted Coverage—CAD Complete) 212. The possible opens 214, possible shorts 216, possible wrong parts 218, possible missing parts 220, possible clocked incorrectly 222, and total possible 244 for the tested device are listed. The testable opens 224, testable shorts 226, testable wrong parts 228, testable missing parts 230, testable clocked incorrectly 232, and total testable 246 for the tested device are listed. The percentage opens 234, percentage shorts 236, percentage wrong parts 238, percentage missing parts 240, percentage clocked incorrectly 242, and total percentage 248 for the tested device are listed. A color code key 250 is provided to show that greater that 70% test coverage is acceptable, between 50% and 70% test coverage is marginal and less than 50% test coverage is poor. The Estimated DPMO 252, Estimated DPU 254, Estimated FPY 256, Estimated Escaping DPU 258 and Estimated Final Yield 260 for the Assembly DPMO is also provided.
FIG. 3 provides Bill of Material (“BOM”) information [0020] 302, mechanical information 304, electrical information 306, probe information 308, test strategy (# of pins) 310, possible defects 312 and testable defects 314 for the device. The Bom information 302 includes the reference designator 320 and part number 322 for the device. The mechanical information 304 includes the package code 324 and the number of pins 326 for the device. The electrical information 306 includes the device type 328 and the device value 330. The probe information 308 includes the number of pins without probes 332 and the number of power pins 334. The test strategy (# of pins) 310 includes RLC 336, parallel tested 338, diode/transistor 340, connection check 342, test jet 344, capacitor check 346, power (analog) 348 and power (digital) 350. The possible defects 312 includes shorts 352, opens 354, wrong part 356, missing part 358 and clocked incorrectly 360. The testable defects 314 includes shorts 362, opens 364, wrong part 366, missing part 368 and clocked incorrectly 370. Each of the rows 372 contain the data for a device.
Although preferred embodiments of the present invention have been described in detail, it will be understood by those skilled in the art that various modifications can be made therein without departing from the spirit and scope of the invention as set forth in the appended claims. [0021]
1. A method for determining a test coverage for a device, the method comprising the steps of:
receiving test data from a test platform or development tool, the test data comprising results of two or more tests on the device;
processing the test data to determine or predict the test coverage for the device; and
storing the test data and test coverage in a standard format.
2. The method as recited in claim 1, further comprising the step of determining the accuracy of the predicted test coverage for the device by comparing the actual and predicted test data.
3. The method as recited in claim 1 wherein the test data comprises a computer readable structure.
4. The method as recited in claim 1 wherein the standard format comprises a database.
5. The method as recited in claim 1 wherein the standard format comprises a file.
6. The method as recited in claim 1 wherein the standard format comprises a report.
7. The method as recited in claim 1 wherein the standard format comprises a spreadsheet.
8. The method as recited in claim 1, further comprising the step of performing two or more tests on the device using the test platform or development tool.
9. The method as recited in claim 1, further comprising the step of exporting the test data and test coverage in the standard format to a database.
10. An apparatus for determining a test coverage for a device comprising:
an interface for extracting test data from a test platform or development tool, the test data comprising results of two or more tests on the device; and
a processor communicably linked to the interface for processing the test data to determine or predict the test coverage for the device; and
a memory communicably linked to the processor for storing the test data and test coverage in a standard format.
11. The apparatus as recited in claim 10, wherein the processor also determines the accuracy of the predicted test coverage for the device by comparing the actual and predicted test data.
12. The apparatus as recited in claim 10 wherein the test data comprises a computer readable structure.
13. The apparatus as recited in claim 10 wherein the standard format comprises a database.
14. The apparatus as recited in claim 10 wherein the standard format comprises a file.
15. The apparatus as recited in claim 10 wherein the standard format comprises a report.
16. The apparatus as recited in claim 10 wherein the standard format comprises a spreadsheet.
17. A system for determining a test coverage for a device comprising:
a test platform or development tool that generates test data, which comprises results of two or more tests on the device;
an interface communicably linked to the test platform or development tool for extracting the test data;
18. The system as recited in claim 17, wherein the processor also determines the accuracy of the predicted test coverage for the device by comparing the actual and predicted test data.
19. The system as recited in claim 17 wherein the test data comprises a computer readable structure.
20. The system as recited in claim 17 wherein the memory comprises a computer readable structure.
21. The system as recited in claim 17 wherein the standard format comprises a database.
22. The system as recited in claim 15 wherein the standard format comprises a file.
23. The system as recited in claim 17 wherein the standard format comprises a report.
24. The system as recited in claim 17 wherein the standard format comprises a spreadsheet.
25. A computer program embodied on a computer readable medium for determining a test coverage for a device comprising:
a code segment for receiving test data from a test platform or development tool, the test data comprising results of two or more tests on the device;
a code segment for processing the test data to determine or predict the test coverage for the device; and
a code segment for storing the test data and test coverage in a standard format.
26. The computer program as recited in claim 25, further comprising a code segment for determining the accuracy of the predicted test coverage for the device by comparing the actual and predicted test data.
27. The computer program as recited in claim 25 wherein the test data comprises a computer readable structure.
28. The computer program as recited in claim 25 wherein the storage structure comprises a computer readable structure.
29. The computer program as recited in claim 25 wherein the standard format comprises a database.
30. The computer program as recited in claim 25 wherein the standard format comprises a file.
31. The computer program as recited in claim 25 wherein the standard format comprises a report.
32. The computer program as recited in claim 25 wherein the standard format comprises a spreadsheet.
33. The computer program as recited in claim 25, further comprising a code segment for exporting the test data and test coverage in the standard format to a database.
US09738068 2000-12-15 2000-12-15 Test coverage analysis system Abandoned US20020078401A1 (en)
US09738068 US20020078401A1 (en) 2000-12-15 2000-12-15 Test coverage analysis system
US20020078401A1 true true US20020078401A1 (en) 2002-06-20
ID=24966431
US09738068 Abandoned US20020078401A1 (en) 2000-12-15 2000-12-15 Test coverage analysis system
US (1) US20020078401A1 (en)
US20070288811A1 (en) * 2004-06-14 2007-12-13 Research In Motion Limited System and method for testing a data storage device without revealing memory content
US20110067005A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method to determine defect risks in software solutions
US20110066890A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method for analyzing alternatives in test plans
US20110067006A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method to classify automated code inspection services defect output for defect analysis
US20110066486A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method for efficient creation and reconciliation of macro and micro level test plans
US20110066893A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method to map defect reduction data to organizational maturity profiles for defect projection modeling
US20110066887A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method to provide continuous calibration estimation and improvement options across a software integration life cycle
US20110066558A1 (en) * 2009-09-11 2011-03-17 International Business Machines Corporation System and method to produce business case metrics based on code inspection service results
US8635056B2 (en) 2009-09-11 2014-01-21 International Business Machines Corporation System and method for system integration test (SIT) planning
US3939588A (en) * 1974-06-24 1976-02-24 Hockaday Robert C Lettering guide apparatus
US4577214A (en) * 1981-05-06 1986-03-18 At&T Bell Laboratories Low-inductance power/ground distribution in a package for a semiconductor chip
US5060052A (en) * 1990-09-04 1991-10-22 Motorola, Inc. TAB bonded semiconductor device having off-chip power and ground distribution
US5068558A (en) * 1988-10-07 1991-11-26 Nippon Ferrofluidics Corporation Magnetic bearing device
US5477160A (en) * 1992-08-12 1995-12-19 Fujitsu Limited Module test card
US5483421A (en) * 1992-03-09 1996-01-09 International Business Machines Corporation IC chip attachment
US5612575A (en) * 1994-05-20 1997-03-18 Matra Marconi Space France Method of connecting the output pads on an integrated circuit chip, and multichip module thus obtained
US5802713A (en) * 1995-01-20 1998-09-08 Fairchild Space And Defense Corportion Circuit board manufacturing method
US5838060A (en) * 1995-12-12 1998-11-17 Comer; Alan E. Stacked assemblies of semiconductor packages containing programmable interconnect
US5838072A (en) * 1997-02-24 1998-11-17 Mosel Vitalic Corporation Intrachip power distribution package and method for semiconductors having a supply node electrically interconnected with one or more intermediate nodes
US5905382A (en) * 1990-08-29 1999-05-18 Micron Technology, Inc. Universal wafer carrier for wafer level die burn-in
US5915752A (en) * 1992-07-24 1999-06-29 Tessera, Inc. Method of making connections to a semiconductor chip assembly
US5936847A (en) * 1996-05-02 1999-08-10 Hei, Inc. Low profile electronic circuit modules
US5942246A (en) * 1996-02-16 1999-08-24 The Liposome Company, Inc. Etherlipid containing multiple lipid liposomes
US6024275A (en) * 1997-06-18 2000-02-15 National Semiconductor Corporation Method of making flip chip and BGA interconnections
US6034332A (en) * 1995-05-22 2000-03-07 Fujitsu Limited Power supply distribution structure for integrated circuit chip modules
US6053395A (en) * 1997-08-06 2000-04-25 Nec Corporation Method of flip-chip bonding between a chip element and a wafer-board
US20020073375A1 (en) * 1997-06-03 2002-06-13 Yoav Hollander Method and apparatus for test generation during circuit design
US20020147939A1 (en) * 1999-01-22 2002-10-10 Andreas Wenzel On-chip debug system with a data band selector
US5929651A (en) * 1995-08-09 1999-07-27 International Business Machines Corporation Semiconductor wafer test and burn-in
US7500160B2 (en) 2004-06-14 2009-03-03 Research In Motion Limited System and method for testing a data storage device without revealing memory content
US8527955B2 (en) * 2009-09-11 2013-09-03 International Business Machines Corporation System and method to classify automated code inspection services defect output for defect analysis
US8566805B2 (en) 2009-09-11 2013-10-22 International Business Machines Corporation System and method to provide continuous calibration estimation and improvement options across a software integration life cycle
US8645921B2 (en) 2009-09-11 2014-02-04 International Business Machines Corporation System and method to determine defect risks in software solutions
US8667458B2 (en) 2009-09-11 2014-03-04 International Business Machines Corporation System and method to produce business case metrics based on code inspection service results
US8924936B2 (en) 2009-09-11 2014-12-30 International Business Machines Corporation System and method to classify automated code inspection services defect output for defect analysis
Bushnell et al. 2004 Essentials of electronic testing for digital, memory and mixed-signal VLSI circuits
Taylor 1981 Quality assurance of chemical measurements
US20060287900A1 (en) 2006-12-21 Inspection system and method of making and using same
US20060174170A1 (en) 2006-08-03 Integrated reporting of data
US6707313B1 (en) 2004-03-16 Systems and methods for testing integrated circuits
Florac et al. 2000 Statistical process control: Analyzing a space shuttle onboard software process
US20020147561A1 (en) 2002-10-10 System and method for intelligent wire testing
US5325309A (en) 1994-06-28 Method and apparatus for integrated circuit diagnosis
US5682392A (en) 1997-10-28 Method and apparatus for the automatic generation of boundary scan description language files
Lavee 1988 Linear structural relationships (LISREL) in family research
Williams 1985 Test length in a self-testing environment
US5668745A (en) 1997-09-16 Method and apparatus for testing of semiconductor devices
Bone et al. 1989 A bootstrap procedure for evaluating goodness-of-fit indices of structural equation and confirmatory factor models
US20040205463A1 (en) 2004-10-14 Apparatus, program, and method for summarizing textual data
Höfer et al. 2007 Status of empirical research in software engineering
US20030188298A1 (en) 2003-10-02 Test coverage framework
US5852796A (en) 1998-12-22 Computerized testing method and system for wire harnesses
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FRY, MICHAEL ANDREW;REEL/FRAME:013717/0500