NIST GCR 15-997

Measuring the PMI Modeling Capability in CAD Systems:

Report 1 - Combined Test Case Verification

Douglas C. Cheney

International TechneGroup Inc.

Bryan R. Fischer

Advanced Dimensional Management LLC

This publication is available free of charge from:

http://dx.doi.org/10.6028/NIST.GCR.15-997
NIST GCR 15-997

Measuring the PMI Modeling Capability in CAD Systems:

Report 1 - Combined Test Case Verification

Prepared for

U.S. Department of Commerce

Engineering Laboratory 

National Institute of Standards and Technology 

Gaithersburg, MD 20899 

By

Douglas C. Cheney 

International TechneGroup Inc. 

Bryan R. Fischer

Advanced Dimensional Management LLC 

This publication is available free of charge from:

http://dx.doi.org/10.6028/NIST.GCR.15-997 

October 2015 

U.S. Department of Commerce 

Penny Pritzker, Secretary

National Institute of Standards and Technology

Willie May, Under Secretary of Commerce for Standards and Technology and Director
i 

Preface

The National Institute of Standards and Technology (NIST) has created a test system to measure 

conformance of Computer-Aided Design (CAD) software to American Society of Mechanical Engineers 

(ASME) standards for product and manufacturing information (PMI), specifically geometric 

dimensioning and tolerancing (GD&T) information. The test system has three main components: test 

cases, test CAD models, and verification and validation test results. The verification and validation 

results measure PMI implementation capabilities in CAD software and derivative STEP, JT, and 3D PDF 

files.

All of the test cases, test models, test results, and other presentations are available from the project 

website: http://www.nist.gov/el/msid/infotest/mbe-pmi-validation.cfm 

This report is the first of three reports about the test system. The reports can be read independently of 

each other.

• Measuring the PMI Modeling Capability in CAD Systems: Report 1 - Combined Test Case Verification

• Measuring the PMI Modeling Capability in CAD Systems: Report 2 - Test Case Validation

• Measuring the PMI Modeling Capability in CAD Systems: Report 3 - Fully-Toleranced Test Case Verification

Disclaimers

The reports were prepared for the Engineering Laboratory of the National Institute of Standards and 

Technology under the following contracts:

• SB1341-12-SE-0860, RECON Services Inc., “PMI Conformance Testing Models”

• SB1341-12-SE-0853, International TechneGroup Inc., “PMI and Composite Information Validation

and Conformance Testing”

• SB1341-14-SE-0061, International TechneGroup Inc., “PMI Test Cases and Models, Validation and

Conformance Testing”

The contents of the reports do not necessarily reflect the views of NIST. NIST and the authors do not 

make any warranty, expressed or implied, nor assume any legal liability or responsibility for the accuracy, 

completeness, or usefulness of any information, product, or process included in the reports. 

Any mention of commercial products is for information purposes only; it does not imply recommendation 

or endorsement by NIST. The test system can be used without any restrictions. Its use in other software 

or hardware products does not imply a recommendation or endorsement by NIST of those products. 

Project Participants

• International TechneGroup Inc. (ITI) - test model creation, expert review, verification, validation, and

documentation

• Advanced Dimensional Management LLC - test case definition and expert review

• RECON Services Inc., Neilsoft Ltd. - test model creation and expert review

• Department of Energy Kansas City Plant (operated by Honeywell FM&T), RECON Services Inc.,

Sigmetrix - expert review

Cover image: Combined test cases
ii

Table of Contents

1 Introduction ........................................................................................................................................... 1

1.1 Model-Based Enterprise and Model-Based Definition .................................................................................. 1

1.2 MBD Verification and Validation ................................................................................................................. 2

1.3 PMI Representation and Presentation............................................................................................................ 2

1.4 PMI Verification and Validation ................................................................................................................... 4

2 Methodology for PMI Modeling Capability Assessment ...................................................................... 5

2.1 Test Case Definition ...................................................................................................................................... 6

2.2 Test Model Creation ...................................................................................................................................... 9

2.3 Test Model Verification...............................................................................................................................12

3 PMI Modeling Capability Results ....................................................................................................... 13

3.1 Representation Limitations..........................................................................................................................17

3.2 Presentation Limitations..............................................................................................................................19

3.3 Style Differences .........................................................................................................................................21

3.4 PMI Verification Challenges.......................................................................................................................22

4 Discussion ............................................................................................................................................ 24

5 References ........................................................................................................................................... 25

Appendix A: PMI Constructs ..................................................................................................................... 27

Appendix B: Combined Test Case Drawings ............................................................................................ 30

Appendix C: Test Model Images ............................................................................................................... 34

Appendix D: Representation Limitation Examples ................................................................................... 42

Appendix E: Presentation Limitation Examples ........................................................................................ 51

Appendix F: Style Difference Examples ................................................................................................... 67
iii

List of Figures

Figure 1: Methodology for PMI modeling capability assessment ................................................................ 5

Figure 2: Typical presentation of a GD&T annotation ................................................................................. 6

Figure 3: Combined test case 1 (CTC 1) drawing......................................................................................... 6

Figure 4: Combined test case 2 (CTC 2) drawing......................................................................................... 7

Figure 5: Combined test case 3 (CTC 3) drawing......................................................................................... 7

Figure 6: Combined test case 4 (CTC 4) drawing......................................................................................... 8

Figure 7: Combined test case 5 (CTC 5) drawing......................................................................................... 8

Figure 8: Combined test case 1 (CTC 1) modeled in CATIA V5 R21 ....................................................... 10

Figure 9: Combined test case 1 (CTC 1) modeled in Creo 2.0 ................................................................... 10

Figure 10: Combined test case 1 (CTC 1) modeled in NX 8.0 ................................................................... 11

Figure 11: Combined test case 1 (CTC 1) modeled in SOLIDWORKS 2012 ............................................ 11

Figure 12: PMI modeling capability results by CAD system ..................................................................... 14

Figure 13: PMI modeling capability results by CAD system, excluding coordinate system structure 

limitations ................................................................................................................................................... 15

Figure 14: Example of a representation limitation ...................................................................................... 17

Figure 15: Example of a presentation limitation ......................................................................................... 19

Figure 16: Example of a style difference .................................................................................................... 21

Figure 17: Extension lines represented as separate dimension annotations ................................................ 22

Figure 18: Different threaded hole supplemental geometry representations .............................................. 23

Figure 19: Different target area representations.......................................................................................... 23

List of Tables

Table 1: Characteristics of PMI representation ............................................................................................ 3

Table 2: Characteristics of PMI presentation ............................................................................................... 3

Table 3: PMI element counts by type and test case .................................................................................... 13

Table 4: PMI representation limitations by characteristic and CAD system .............................................. 15

Table 5: PMI presentation limitations by characteristic and CAD system ................................................. 16

Table 6: PMI entity abbreviations ............................................................................................................... 17

Table 7: Representation limitation counts by characteristic and type ......................................................... 18

Table 8: Presentation limitation counts by characteristic and type ............................................................. 20

Table 9: Style difference counts by characteristic and type ........................................................................ 22
1 

1 Introduction

A methodology for measuring the product and manufacturing information (PMI) modeling capability of 

computer-aided design (CAD) systems has been developed to measure technology readiness and to track 

progress as functionality gaps are closed. A measurement methodology will enhance the ability of

discrete-part manufacturing companies to implement a model-based enterprise (MBE) [1-5]. The use of a 

clear capability assessment will accelerate MBE technology development by CAD software vendors. 

This can increase the business opportunities for both manufacturing companies and technology providers. 

Common practice in discrete-part manufacturing companies is to use CAD systems to create threedimensional (3D) models that precisely define the shape of their products. The companies derive twodimensional (2D) drawings from the 3D model that detail the product’s dimensions, tolerances, and other 

manufacturing information. Manufacturing organizations have typically considered the drawings to be 

the master product definition for all downstream processes such as simulation, manufacturing, and 

inspection. Often a 3D model is recreated from the drawing in one or more downstream processes, 

especially when performed by external suppliers. In some cases, the original 3D model is released with 

the drawing as a reference document [6]. 

As the drawing goes through several engineering changes, the 3D model may become outdated because it 

is not the master design document. Therefore, model recreation from the drawing tends to increase as a 

product matures. Downstream consumers of the drawing visually interpret the dimensions, tolerances, 

and other manufacturing information and manually reenter this information into downstream systems. 

Manually reentering information is a potentially error-prone process. This human interpretation is 

repeated for each engineering change.

Global business requirements are driving companies to produce better and cheaper products in less time 

to market. Management initiatives target the reduction of risk due to variation and the elimination of all 

non-value-added tasks throughout the engineering, manufacturing, and sustainment phases of a product’s 

lifecycle. A leading process improvement initiative today is the concept of MBE [7]. 

1.1 Model-Based Enterprise and Model-Based Definition

A model-based enterprise (MBE) builds on the foundation that all product data may be integrated into a 

single model-based definition (MBD). This eliminates the need for 2D drawing generation, the recreation 

of 3D models, and/or the visual interpretation of drawing data in downstream processes. It improves 

product quality by eliminating drawing-to-model inconsistencies, unintentional model changes during 

recreation, and drawing interpretation errors. It decreases overall time to market by enabling direct reuse 

of the digital product model in downstream software systems.

A key component of an MBD is the integration of all the product and manufacturing information (PMI) 

into the 3D model. Dimensions, tolerances, notes, and other data previously found on a drawing are 

displayed in the model with direct links to the affected portion of the model’s shape definition or 3D 

geometry. The data is grouped into multiple saved views to aid visual consumption. More importantly,

the visual data is linked to an internal representation that is well defined and structured for automated 

consumption in downstream software systems. Derivative models, such as STEP (ISO 10303 –known 

informally as the STandard for Exchange of Product model data) [8-10], JT [11-13] and 3D PDF [14-16] 

files, are created as needed for downstream consumers who do not have direct access to the CAD system 

in which the native MBD model is defined. 
2

1.2 MBD Verification and Validation

In a drawing-based product lifecycle, the drawing is manually checked by a person before release and 

then visually interpreted by a person during downstream reuse. This results in processes that tolerate lowlevel variation in the digital data while being fairly controlled. In a model-based process, the checking 

task is often eliminated on the assumption that a precise native model should be directly reusable in 

downstream systems without error. This results in processes that are less tolerant of digital data variation 

while being less controlled. However, if a company is going to rely on an MBD model throughout its 

product’s lifecycle, the model must be reliable. Therefore, quality checking of the geometry and PMI in 

the master model, and their equivalent entities in all derivatives, is critical before release to downstream 

processes.

Various automotive, aerospace, and defense industry groups have identified precise geometry and PMI 

quality criteria for native MBD models and their derivatives. These include:

• Strategic Automotive Special Interest Group (SASIG) Product Data Quality (PDQ) team [17]

• PDES, Inc. [18] and ProSTEP iViP [19] collaboration for Long-Term Archival (LOTAR) [20]

• Department of Defense’s MBE team [1]

Each group has recently documented these requirements in international, regional, and domestic standards 

such as:

• Managed Model-based 3D Engineering - STEP ISO 10303-242 [21, 22]

• CAD mechanical 3D Explicit geometry information - EN9300-110 [23]

• DoD Standard Practice: Technical Data Packages - MIL-STD-31000A [24]

These groups generally agree that the process of quality checking a native CAD model should be called 

verification. This process verifies that the product definition data is complete, consistent, and conformant 

to relevant standards. They recommend that the process of determining whether the data in a derivative 

model is equivalent to the native model should be called validation. This process validates that all data 

has been translated with any digital variation within acceptable limits specified by the anticipated 

downstream processes.

Due to the complexity of MBD data, it is unrealistic to implement verification or validation using an 

interactive, manual process. Several CAD applications have been developed to automate verification and 

validation using the criteria referenced above. While these applications make MBD quality control 

feasible, they impose an important requirement on the CAD modeling systems: that all MBD data, 

including 3D geometry and PMI, must be accessible through an application programming interface (API) 

to third-party developers.

1.3 PMI Representation and Presentation

An MBD must contain sufficient PMI representation so that automated systems, such as machining and 

inspection, can reuse the information efficiently and correctly in all downstream processes. PMI 

representation (also known as semantic PMI) includes all information necessary to represent GD&T 

without any graphical presentation elements. The PMI presentation should also be clearly presented for 

visual (human) consumers so that they understand and trust the model-based definition. PMI presentation 

(also known as graphical PMI) consists of geometric elements such as lines and arcs preserving the exact 

appearance (color, shape, positioning) of the GD&T annotations. The internal PMI representation should 

be structured and defined so each element is clear, complete, and consistent. The PMI presentation 
3

should be organized into saved views with annotations that support cross-highlighting of affected 

geometry. 

These two aspects of PMI, representation and presentation, are best understood by considering how their 

key characteristics are applied to the various components of an MBD. Table 1 and Table 2 list the 

characteristics of PMI representation and presentation, respectively. The following is an explanation how 

they apply to the product geometry, coordinate systems, supplemental geometry, annotations, and saved 

views in an MBD.

Table 1: Characteristics of 

PMI representation

Table 2: Characteristics of 

PMI presentation

MBD product geometry is structured to differentiate the geometric entities that define the 3D shape of the 

product from other entities used as reference, context, or supplemental geometry for annotations. For 

most discrete-part product models, a solid (closed volume) or shell (open surface) definition provides the 

highest level of definition for downstream processes. The parametric definition of the model is complete, 

correct, and useful for revisioning. The explicit definition of topology and geometry is free of defects that 

impede downstream reuse. The meta-data properties associated with the product model capture basic 

product management data, such as ownership and lifecycle state. The visibility status and display color of 

the product geometry are appropriate for visual interpretation by downstream users.

MBD annotations have a specified type (dimension, feature control frame, note, etc.) and named 

parameters (nominal value, tolerance, material modifier, etc.) that facilitate automated interpretation 

downstream. An annotation’s associated geometry includes all affected surfaces in the product geometry 

and any supplemental geometry. It does not include any extraneous geometry. This facilitates both 

automated consumption and visual interpretation, also known as cross-highlighting. The visibility, 

layout, location, and orientation of the annotation in saved views, along with its color, display name, 

lines, and text, are appropriate for visual interpretation by downstream users.

MBD coordinate systems have explicit named associations with the feature control frames that rely on the 

datum reference frames they represent. Each coordinate system’s location and orientation accurately

represent the datum reference frame. The coordinate system’s visibility in each saved view corresponds 
4

to the visibility of its associated annotations. Its color, name, and display text are appropriate for visual 

interpretation by downstream users.

Supplemental geometry is geometric elements that do not belong to the shape of a part. The geometric 

elements are used to create other shapes or contain information about part features such as hole 

centerlines. MBD supplemental geometry entities have the correct form or structure for the annotations

that references them. For example, the limited area for a datum target defines the portion of the 

underlying solid face or surface that is inside versus outside. The location, orientation, and size of each

supplemental geometry entity complete the conceptual definition of its associated annotations. Its 

visibility in saved views corresponds to the visibility of its associated annotations. Supplemental

geometry color is appropriate for visual interpretation by downstream users. 

A saved view facilitates the presentation of the model and associated PMI by defining a subset of the PMI 

and an orientation from which it is viewed. MBD saved views are structured to contain a related set of 

annotations, with their associated supplemental geometry and coordinate systems, along with the 

appropriate product geometry. Each saved view may contain the complete geometric definition of the 

product or a portion defined by a cross section. The contents of a saved view are displayed within a 

frustum, or pyramid of vision, that is intuitive for visual interpretation by downstream users.

1.4 PMI Verification and Validation

The process of querying PMI data in an MBD model for verification is straightforward as long as the 

CAD API provides sufficient access to the data. First, the type and properties of each annotation entity 

are retrieved and compared with those specified in the test case documentation. Second, any relationships 

between the annotation and other annotations or geometry entities are queried and compared with the 

specification. Since an MBD model may contain multiple annotations with similar types and properties, 

it may be necessary also to query the graphic presentation data in order to match reliably each annotation 

with its specification and to confirm its relationships are correct.

The process of comparing PMI constructs between MBD models in dissimilar CAD systems for 

equivalence validation is more complex. The primary challenge is to correctly match corresponding 

annotation entities before comparing their characteristics. Because all of the presentation characteristics 

can vary significantly without changing the meaning (representation), these cannot be reliably used for 

matching purposes. The test model images in Figures 8-11 illustrate the typical variation between the 

CAD systems used for this assessment. Reliable annotation matching requires that all product and 

supplemental geometry entities be matched. Then, the subset of annotations entities associated with each 

set of matching geometry entities are matched and compared. Annotations that have been added, 

removed, or had their geometry associations changed will remain unmatched.

Some PMI constructs make automation of the above verification and validation processes difficult (see 

section 3.4). The various CAD systems use different modeling methodologies for these constructs that 

are each considered valid within the ASME standards. Until the CAD systems converge toward common 

methodologies, or the standards are modified to require this, the MBD verification and validation 

technologies must implement advanced reasoning and exception handling to accommodate this allowable 

variation in PMI definition.
5

2 Methodology for PMI Modeling Capability Assessment

The PMI modeling capability of the CAD systems commonly used by discrete-part manufacturing 

companies to support MBE was assessed using a formal methodology [25], shown in Figure 1, involving:

1. Test case definition and expert review

2. Test CAD model creation based on the test case definitions

3. Verification of the CAD models against the test case definitions

4. Generation of derivative STEP, JT, and 3D PDF files by the Implementor Forums [12, 14, 26]

5. Validation of the derivative files against the CAD models and test case definitions

This report is concerned with steps 1-3 of the PMI modeling capability assessment. The validation of 

derivative files for one of the test cases is documented in second report of this series [27]. The 

verification of other test cases is documented in the third report of this series [28].

Figure 1: Methodology for PMI modeling capability assessment
6

2.1 Test Case Definition

For test case generation, an industry expert in geometric dimensioning and tolerancing (GD&T) defined

representative PMI constructs allowed by the American Society of Mechanical Engineers (ASME)

standards for 2D drawings Y14.5-1994 [29] and 3D models Y14.41-2003 [30]. (Newer versions of both 

standards are available.) A PMI construct is a group of annotation entities which define an elemental 

concept, for example: defining a datum feature with a datum feature symbol (one annotation) or 

controlling the variation of a hole with a size dimension, a feature control frame, and its associated datum

features (3 to 5 annotations). Figure 2 shows the presentation of a typical GD&T annotation [31].

Figure 2: Typical presentation of a GD&T annotation

The constructs defined for this assessment are listed in Appendix A. The constructs were applied to five

discrete-part geometry models, with approximately ten PMI constructs in each model.

Each combined test case (CTC) is documented with a set of drawings and explanatory text, as shown in 

Figures 3-7. Drawings of other views of each test case are in Appendix B.

Figure 3: Combined test case 1 (CTC 1) drawing
7

Figure 4: Combined test case 2 (CTC 2) drawing

Figure 5: Combined test case 3 (CTC 3) drawing
8

Figure 6: Combined test case 4 (CTC 4) drawing

Figure 7: Combined test case 5 (CTC 5) drawing
9

Other industry GD&T experts reviewed the five CTCs for clarity and correctness. The CTCs were 

refined based on the expert feedback. All experts agreed that the CTCs are intended to simply combine

representative constructs and do not define products that are fully-toleranced and/or functional for 

tolerance purposes. The test cases are also not intended to represent best practice in how to apply GD&T 

to a part. Simpler GD&T strategies could have been used. The test cases are intended to exercise valid 

presentations of GD&T defined in the ASME Y14 standards.

2.2 Test Model Creation

A team of CAD experts created CAD models for each CTC in four CAD systems that were available in

early 2013:

• CATIA V5 R21 from Dassault Systemes [32]

• Creo 2.0 from PTC [33]

• NX 8.0 from Siemens PLM [34]

• SOLIDWORKS 2012 from Dassault Systemes [35]

The CAD experts used the above PMI representation and presentation criteria to create models with 

equivalent meaning, and negligible graphical variation. When it was not possible to satisfy both sets of 

criteria, the representation was given precedence over the presentation. Figures 8-11 show combined test 

case 1 (CTC 1) modeled in each of the four CAD systems. Images of each test model, each with multiple 

saved views, are shown in Appendix C.
10

Figure 8: Combined test case 1 (CTC 1) modeled in CATIA V5 R21

Figure 9: Combined test case 1 (CTC 1) modeled in Creo 2.0
11

Figure 10: Combined test case 1 (CTC 1) modeled in NX 8.0

Figure 11: Combined test case 1 (CTC 1) modeled in SOLIDWORKS 2012
12

2.3 Test Model Verification

The CAD validation software CADIQ 8.0 [36] was used to query the PMI representation and presentation

data in a 3D model using the API of each CAD system. The software vendor for CADIQ developed and 

refined algorithms for matching and comparing each data element between models in different CAD 

systems that were based on the same test case definition.

After the models were complete, a CAD validation specialist manually compared the data queried for

each PMI element in the five models for one CAD system to the five test case definitions. Significant 

discrepancies or deficiencies were documented. Once the CAD modeling team resolved the identified 

issues in the models, the data set was designated as the reference set. Using the multi-CAD PMI 

validation technology, the specialist automatically compared each model from the other three CAD 

systems to the reference model.

Each discrepancy between the PMI in a model pair was compared with the test case to determine which 

model was inconsistent. Then, interactive CAD system queries were used to determine whether the 

discrepancy was due to measurement error in the validation tool or a difference in the test model. The 

validation software vendor resolved measurement errors while the CAD modeling team resolved model 

discrepancies within the limitations of the CAD system.

After several iterations of model refinement and verification, the outstanding discrepancies were 

documented as system limitations and the test models were released to the CAD software vendor 

representatives in the CAx Implementor Forum (CAx-IF) [26] for review. The CAD software vendors 

provided additional feedback to resolve any outstanding modeling issues.
13

3 PMI Modeling Capability Results

The testing methodology was used to determine whether the representation and presentation of each PMI 

element (i.e., annotation, coordinate system, supplemental geometry entity, saved view) in each test 

model were well defined. The PMI element counts for this representative data set are shown in

Table 3.

Table 3: PMI element counts by type and test case

All PMI elements with a representation limitation were counted, by element type, across all test models 

for each CAD system. These counts were used to calculate a “Representation Limitation” percentage 

using this formula:

LLLLLLLLLLLLLLLLLLLL PPPPPPPPPP PPPP = 100 xx

LLLLLLLLLLLLLLLLLLLL CC CC

EEEE CC CC

All PMI elements with only a presentation limitation were counted and likewise divided by the element 

count to produce a “Presentation Limitations Only” percentage. If an element had both a representation 

and a presentation limitation, it was included only in the representation percentage. If an element had two 

or more representation and/or presentation limitations, it was counted only once in the appropriate 

calculation. Elements with neither type of limitation were counted in a “No Limitations” percentage, 

thus:

NN LLLLLLLLLLLLLLLLLLLLLL = 100% − (RR RR LLLLLLLLLLLLLLLLLLLLLL + PPPPPP PP LLLLLLLLLLLLLLLLLLLLLL)

These three modeling capability percentages for each CAD system are shown in Figure 12. The names of 

the CAD systems have been generalized to give the end-user community an overall summary of their 

capabilities without impugning any particular CAD vendor. The technical details have been shared 

separately with each CAD vendor so they know their opportunity for improvement in the MBE domain.
14

Figure 12: PMI modeling capability results by CAD system

In Figure 12, the “No Limitations” percentage can be interpreted as a measure of the capability of the 

CAD system to satisfy both the automated and visual consumption requirements of downstream MBE 

processes relative to the functional coverage of PMI constructs of this set of test cases. The 

“Representation Level” percentage, calculated as 100% less the “Representation Limitations” percentage,

indicates the CAD system’s ability to satisfy only automated consumption requirements.

The representation and presentation limitations for each CAD system were then subtotaled by 

characteristic and divided by the count of PMI elements of the type appropriate for that characteristic 

using this formula:

VV VV PPPPPPPPPP PPPP = 100 xx

EEEE CC CC − LLLLLLLLLLLLLLLLLLLL CC CC

EEEE CC CC

For example, the count of annotation structure limitations for all models in each CAD system was divided 

by the count of annotations in the test case using the above formula. The verification percentages for 

each element type in each CAD system are shown in Table 4 and Table 5.

Table 4 reveals that all CAD systems failed to represent the expected structure of the specified coordinate 

systems. The remaining representation limitations were limited to annotations, although the specific 

percentages vary slightly.
15

Table 4: PMI representation limitations by characteristic and CAD system

Because the coordinate system structure limitations were consistent across all CAD systems, thus creating 

a uniform bias, it is useful to consider an adjustment to the overall statistics in Figure 12 that excludes all 

coordinate system limitations. Figure 13 shows these adjusted statistics.

Figure 13: PMI modeling capability results by CAD system, 

excluding coordinate system structure limitations

Table 5 shows a much broader variation in the types of presentation limitations across CAD systems.

Some of the systems were unable to adequately present coordinate system and saved view characteristics, 

which accounts for their larger overall “Presentation Limitations only” percentages relative to the other 

systems shown in Figure 13.
16

Table 5: PMI presentation limitations by characteristic and CAD system
17

3.1 Representation Limitations

For each characteristic, there were often multiple types of limitations. Appendix D shows one example 

of each type of PMI representation limitation. The graphics in the appendices have been generalized to 

avoid identifying the specific CAD system involved. Figure 14 shows one example from Appendix D.

Table 7 tabulates the count of representation limitations by characteristic and type across all CAD 

systems. Table 6 explains the PMI entity abbreviations used in Table 7.

Figure 14: Example of a representation limitation

Table 6: PMI entity abbreviations
18

Table 7: Representation limitation counts by characteristic and type
19

3.2 Presentation Limitations

Appendix E shows one example of each type of presentation limitation. Figure 15 shows one example 

from Appendix E. Table 8 tabulates the count of representation limitations by characteristic and type 

across all CAD systems. Table 6 explains the PMI entity abbreviations used in Table 8.

Figure 15: Example of a presentation limitation
20

Table 8: Presentation limitation counts by characteristic and type
21

3.3 Style Differences

In some cases, the representation and presentation for a PMI element were determined by the expert 

reviewers to be correct yet different between the CAD systems. These variations were categorized as 

style differences and not included in the representation or presentation limitation calculations. Appendix F

documents one example of each type of style difference that was ignored. Figure 16 shows an example 

from Appendix F. Table 9 tabulates the count of style differences by characteristic and type across all 

systems. Table 6 explains the PMI entity abbreviations used in Table 9.

Figure 16: Example of a style difference
22

Table 9: Style difference counts by characteristic and type

3.4 PMI Verification Challenges

A challenging construct is the representation of extension lines for datum feature symbols and feature 

control frames. In some CAD systems, this construct is represented as dimension entities that are separate 

from the attached annotation, as shown in Figure 17. These extra annotations introduce parameters 

(nominal value and limits) that must be ignored during verification.

Figure 17: Extension lines represented as separate dimension annotations
23

Another challenging construct is the representation of threaded holes shown in Figure 18. The diameter of 

the simple hole in the solid model may be different for the same type of threaded hole in various CAD 

systems. Some systems use differing supplemental geometry, such as wireframe curves or non-solid 

surfaces, to represent the hole thread depth while others use no supplemental geometry.

Figure 18: Different threaded hole supplemental geometry representations

Finally, when a PMI construct is specified with a limited area, such as a datum target or geometric 

tolerance, the portion of the product shapes that is within the target area is represented differently. Some 

CAD systems define a non-solid surface overlaid on the solid while others subdivide the portion of the 

solid face into a separate face shown in Figure 19. Still others indicate the area with a region defined by 

wireframe geometry. These modeling differences create significant variability that must be accounted for 

during annotation matching and comparison.

Figure 19: Different target area representations
24

4 Discussion

Using a formal methodology, implemented with advanced verification and validation technology, the 

MBE modeling capability of four leading CAD systems was quantified relative to the PMI requirements 

captured in five combined test cases.

The four CAD systems, which were tested at 2013 release levels, are roughly equivalent in their PMI 

representation modeling capability (between 81% and 87%) for the functionality within scope of this 

assessment (between 89% and 95% when adjusted for the consistent coordinate system limitation). Their 

capability measurements vary much more (between 67% and 85%) when both automated and visual 

consumption requirements are considered.

The specific PMI representation and presentation system limitations identified by this assessment have 

been clearly documented and communicated to the CAD vendors.

The specific test of the PMI capabilities in CAD systems documented in this report is a snapshot in time. 

Specific test cases were developed using particular versions of the ASME Y14 tolerancing standards and 

PMI constructs. The test cases were modeled in particular versions of four CAD systems with a specific 

modeling methodology to give precedence to PMI representation over PMI presentation. The CAD 

models were compared to each other with a particular version of CAD validation software. Results for 

PMI representation and presentation capabilities were reported based on four categories of PMI elements: 

annotations, coordinate systems, supplemental geometry, and saved views.

For a company that is transitioning from 2D drawings to 3D models to implement model-based design, 

this report can be used to identify the characteristics of PMI representation and presentation and the 

capabilities of CAD software that are important to achieve an MBD workflow. The test cases may or 

may not be representative of the types of PMI that might be typically used. The versions of the CAD 

systems and tolerancing standards might be newer or older than what a company requires. However, the 

report clearly identifies a wide variety of PMI representation and presentation issues that can be used to 

evaluate CAD software that is used in an MBD environment. 
25

5 References

[1] Model Based Enterprise - Exploring the Digital Tapestry, http://www.model-basedenterprise.org/.

[2] J. Lubell, K. Chen, J. Horst, S. Frechette, and P. Huang, "Model Based Enterprise / Technical 

Data Package Summit Report," NIST Technical Note 1753, 2012.

[3] J. Lubell, S. P. Frechette, R. R. Lipman, F. M. Proctor, J. A. Horst, M. Carlisle, and P. J. Huang, 

"Model-Based Enterprise Summit Report," National Institute of Standards and Technology, NIST 

Technical Note 1820, 2013.

[4] Model-Based Enterprise Summit 2013, National Institute of Standards and Technology, 

http://www.nist.gov/el/msid/mbe_2013_presentations.cfm.

[5] Model-Based Enterprise Summit 2014, National Institute of Standards and Technology, 

http://www.nist.gov/el/msid/mbe2014presentations.cfm.

[6] V. Quintana, L. Rivest, R. Pellerin, F. Venne, and F. Kheddouci, "Will Model-based Definition 

replace engineering drawings throughout the product lifecycle? A global perspective from 

aerospace industry," Computers in Industry, vol. 61, pp. 497-508, 2010.

[7] M. Alemanni, F. Destefanis, and E. Vezzetti, "Model-based definition design in the product 

lifecycle management scenario," The International Journal of Advanced Manufacturing 

Technology, vol. 52, pp. 1-14, 2011.

[8] ISO 10303-1:1994, "Industrial automation systems and integration - Product data representation 

and exchange - Part 1: Overview and fundamental principles," International Organization for 

Standardization, Geneva, Switzerland.

[9] M. J. Pratt, "Introduction to ISO 10303—the STEP standard for product data exchange," Journal 

of Computing and Information Science in Engineering, vol. 1, pp. 102-103, 2001.

[10] STEP Application Handbook, ISO 10303, Version 3, SCRA, 2006, 

https://pdesinc.org/downloadable_files/STEPapplicationhandbook63006BF.pdf.

[11] JT Open, Siemens, http://www.plm.automation.siemens.com/en_us/products/open/jtopen/.

[12] JT Implementor Forum, ProSTEP iViP, http://www.prostep.org/en/projects/jt-implementorforum.html.

[13] ISO 14306:2012, "Industrial automation systems and integration - JT file format specification for 

3D visualization," International Organization for Standardization, Geneva, Switzerland.

[14] 3D PDF Consortium, http://www.3dpdfconsortium.org/.

[15] ISO 24517-1:2008, "Document management -- Engineering document format using PDF -- Part 

1: Use of PDF 1.6 (PDF/E-1)," International Organization for Standardization, Geneva, 

Switzerland.

[16] ISO 14739-1:2014, "Document management -- 3D use of Product Representation Compact (PRC) 

format -- Part 1: PRC 10001," International Organization for Standardization, Geneva, 

Switzerland.

[17] ISO/PAS 26183:2006, "SASIG Product data quality guidelines for the global automotive 

industry," International Organization for Standardization, Geneva, Switzerland.

[18] PDES, Inc. - Advancing the Digital Enterprise, https://pdesinc.org/.

[19] ProSTEP iViP Association, http://prostep.org/.

[20] LOTAR - Long Term Archiving and Retrieval, http://www.lotar-international.org/.

[21] Development of a Convergent Modular STEP Application Protocol Based on AP 203 and AP 

214: STEP AP 242 – Managed Model Based 3D Engineering, ASD Strategic Standardization 

Group, 2009, http://www.ap242.org/.

[22] ISO 10303-242:2014, "Industrial automation systems and integration - Product data 

representation and exchange - Part 242: Application protocol: Managed Model-based 3D 

Engineering," International Organization for Standardization, Geneva, Switzerland.

[23] ASD Strategic Standardization Group - Long Term Archiving and Retrieval - LOTAR, 

http://www.asd-ssg.org/lotar.
26

[24] MIL-STD-31000A, "DoD Standard Practice: Technical Data Packages," U.S. Department of 

Defense, 2013.

[25] S. P. Frechette, A. T. Jones, and B. R. Fischer, "Strategy for Testing Conformance to Geometric 

Dimensioning & Tolerancing Standards," Procedia CIRP, vol. 10, pp. 211-215, 2013.

[26] CAx Implementor Forum, http://www.cax-if.org/.

[27] D. Cheney and B. Fischer, "Measuring the PMI Modeling Capability in CAD Systems: Report 2 -

Test Case Validation," National Institute of Standards and Technology, NIST-GCR 15-998, 2015.

[28] D. Cheney and B. Fischer, "Measuring the PMI Modeling Capability in CAD Systems: Report 3 -

Fully-Toleranced Test Case Verification," National Institute of Standards and Technology, NISTGCR 15-999, 2015.

[29] ASME Y14.5-1994, "Dimensioning and Tolerancing - Engineering Drawing and Related 

Documentation Practices," American Society of Mechanical Engineers, 1994, New York.

[30] ASME Y14.41-2003, "Digital Product Definition Data Practices - Engineering Drawing and 

Related Documentation Practices," American Society of Mechanical Engineers, 2003, New York.

[31] R. Lipman and J. Lubell, "Conformance checking of PMI representation in CAD model STEP 

data exchange files," Computer-Aided Design, vol. 66, pp. 14-23, 2015.

[32] CATIA, Dassault Systemes, http://www.3ds.com/products-services/catia/.

[33] Creo, PTC, http://www.ptc.com/product/creo.

[34] NX, Siemens, www.plm.automation.siemens.com/en_us/products/nx/.

[35] SOLIDWORKS, Dassault Systemes, http://www.solidworks.com/.

[36] CADIQ, ITI TranscenData, http://www.transcendata.com/products/cadiq/.
27

Appendix A: PMI Constructs
28
29
30

Appendix B: Combined Test Case Drawings
31
32
33
34

Appendix C: Test Model Images

Combined Test Case 1

Saved View MBD_0

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
35

Combined Test Case 2

Saved View MBD_A

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
36

Combined Test Case 2

Saved View MBD_B

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
37

Combined Test Case 2

Saved View MBD_C

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
38

Combined Test Case 3

Saved View MBD_0

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
39

Combined Test Case 4

Saved View MBD_0

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
40

Combined Test Case 5

Saved View MBD_A

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
41

Combined Test Case 5

Saved View MBD_B

• Clockwise from upper left - Test Models for CATIA V5 R21, NX 8.0, SOLIDWORKS 2012 and 

Creo 2.0

• Screenshot is of each test model displayed in CADIQ

• Annotations and their associated geometry are highlighted in red
42

Appendix D: Representation Limitation Examples
43
44
45
46
47
48
49
50
51

Appendix E: Presentation Limitation Examples
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

Appendix F: Style Difference Examples
68
69
70
71
72