YOUR CENTRAL SOURCE FOR DATA EXCHANGE

www.TranscenData.com

NIST MBE PMI Validation & Conformance Testing

CTC Model Verification Results

February 2015

Doug Cheney 

CAD Validation Specialist

ITI TranscenData

Doug.Cheney@TranscenData.com 

Any mention of commercial products is for information only; it does not imply a recommendation or endorsement by NIST.
NIST is Supporting the Discrete Part Manufacturing 

Industry for Long-term Growth

2 NIST MBE PMI CTC Model Verification Results

Manuf.

Industry

PLM 

Software

Vendors

MBE 

Test Data &

Validation 

Testing

High-Quality

MBE 

Software

“Our strategic direction is the 

Model-Based Enterprise (MBE)”

MBE 

Reference 

Data

MBE 

Research

Partnership
NIST MBE PMI Validation and Conformance Testing 

Program Objectives

Develop test cases,

test models and

software algorithms sufficient to

measure conformance of CAD systems to 

American Society of Mechanical Engineers (ASME) 

standards for Product and Manufacturing Information 

(PMI). 

3 NIST MBE PMI CTC Model Verification Results

ASME Y14.5 Dimensioning and Tolerancing

ASME Y14.41 Digital Product Data Definition Practices 

Web site: http://go.usa.gov/mGVm
Example Combined Test Case (CTC) and Test Models

4 NIST MBE PMI CTC Model Verification Results

Test Case CATIA V5 R21 Model

NX 8.0 Model

Creo 2.0 Model

SolidWorks 2012 Model
Conformance Testing Terminology

PMI element: An annotation, coordinate system,

supplemental geometry entity, or saved view

Presentation: What the human user sees

(Visual consumption)

Representation: What the downstream software receives

(Automated consumption)

Verification: How well each PMI element is modeled

(CAD system capability)

Validation: How well each PMI element is translated

(Translator capability)

5 NIST MBE PMI CTC Model Verification Results
Verification Testing Methodology

The representation and presentation of each PMI element is 

compared to the test case specification

Any differences, which cannot be resolved with an alternate 

modeling technique, are categorized as a:

– Representation Limitation

– Presentation Limitation

– Style Difference (representation and presentation are correct but 

different between systems)

Each limitation/difference is grouped by characteristic and 

type

An example of each characteristic-type combination is 

documented in this presentation

The following slide indexes (underlined) these examples

6 NIST MBE PMI CTC Model Verification Results
Verification Characteristics (and index to examples)

Representation Limitation

Annotation structure

Annotation parameters

Annotation geometry

Coordinate system structure

Coordinate system parameters

Supplemental geometry structure

Supplemental geometry parameters

Style Difference

Annotation structure

Annotation geometry

Supplemental geometry structure

Product geometry parameters

Presentation Limitation

Annotation visibility

Annotation color

Annotation name

Annotation layout

Annotation location

Annotation orientation

Annotation lines

Annotation text

Coordinate system visibility

Coordinate system color

Coordinate system name

Coordinate system text

Supplemental geometry visibility

Supplemental geometry color

Saved view structure

Saved view name

Saved view frustum

7 NIST MBE PMI CTC Model Verification Results

If a characteristic is not underlined,

no limitations were found in this dataset.
Verification Capability Measurement Methodology

Each PMI element limitation/difference is counted by 

category, characteristic, and type

The subtotal per characteristic is divided by the subtotal of 

PMI elements to which it applies

– Multiple limitations/differences of the same characteristic for the same 

PMI element are only counted once

The total per category is divided by the total PMI elements

– Multiple limitations/differences of the same category for the same PMI 

element are only counted once, with representation > presentation

The following slides show these statistics for this dataset

The name of each CAD system is generalized according to 

the overall results (“CAD A” better than “CAD B”...)

8 NIST MBE PMI CTC Model Verification Results
Representation Limitation and Style Difference Counts 

by Characteristic and Type (across all systems)

9 NIST MBE PMI CTC Model Verification Results
Presentation Limitation Counts

by Characteristic and Type (across all systems)

10 NIST MBE PMI CTC Model Verification Results
Verification Percentages by Characteristic and System

11 NIST MBE PMI CTC Model Verification Results

Each percentage is calculated using 

this ratio:

(Element count – Limitation count)

Element count

The limitation count excludes limitations 

with the same characteristic 

(but different types) on the same PMI 

element.

Percentages less than 90% are shown 

in bold font.
Verification Percentages by System

12 NIST MBE PMI CTC Model Verification Results

“No Limitations” measures the capability for 

both automated and visual consumption and is 

calculated as 100% less the other limitation 

percentages.

“Representation Level” measures the capability 

for automated consumption only and is 

calculated as 100% less the representation 

limitations percentage.

The presentation and representation limitation 

percentages are calculated using this ratio:

Limitation count

Element count

The “Presentation Limitation Only” count 

excludes presentation limitations that overlap 

with representation limitations on the same PMI 

element.

The “Element Count” includes all annotations, 

coordinate systems, supplemental geometry 

entities, and saved views specified in the test 

cases.
Verification Percentages by System - Adjusted

13 NIST MBE PMI CTC Model Verification Results

Because all of these CAD systems had the 

same coordinate system structure limitation 

(not linked to the datum reference frames), 

these are the overall verification percentages 

if coordinate system issues are excluded.
Verification Summary

The limitation characteristics and types are different for each 

system.

All 4 CAD systems are able to represent more than 90% of 

the annotations, supplemental geometry entities, and saved 

views in this verification test.

None of the systems are able to represent the expected 

coordinate system structure.

The systems vary significantly in their ability to visually 

present the PMI elements as specified in the test cases.

None of the systems are able to present the complex 

dimensions (counterbore, countersink, threaded hole) as 

specified in the test case while representing the correct 

geometry associations for each component of the dimension.

14 NIST MBE PMI CTC Model Verification Results
Verification Characteristics (and index to examples)

Representation Limitation

Annotation structure

Annotation parameters

Annotation geometry

Coordinate system structure

Coordinate system parameters

Supplemental geometry structure

Supplemental geometry parameters

Style Difference

Annotation structure

Annotation geometry

Supplemental geometry structure

Product geometry parameters

Presentation Limitation

Annotation visibility

Annotation color

Annotation name

Annotation layout

Annotation location

Annotation orientation

Annotation lines

Annotation text

Coordinate system visibility

Coordinate system color

Coordinate system name

Coordinate system text

Supplemental geometry visibility

Supplemental geometry color

Saved view structure

Saved view name

Saved view frustum

15 NIST MBE PMI CTC Model Verification Results

If a characteristic is not underlined,

no limitations were found in this dataset.
Return

to Index

CAD System Representation Limitations for

Annotation Structure

16 NIST MBE PMI CTC Model Verification Results
Representation Limitation

Return

to Index

Annotation Structure:

Countersink diameter DIM not defined

17 NIST MBE PMI CTC Model Verification Results

The diameter dimension for the outside (larger, circular) edge 

of this countersink is not defined.
Representation Limitation

Return

to Index

Annotation Structure:

DIM defined as part of DTS

18 NIST MBE PMI CTC Model Verification Results

The dimensions for the target area of this datum target 

are represented as parameters in the datum target 

symbol and not as separate dimensions.

Test Case
Representation Limitation

Return

to Index

Annotation Structure:

FCF extension line defined as separate DIM

19 NIST MBE PMI CTC Model Verification Results

The extension lines for this feature control frame, whose 

orientation is critical to the tolerance zone definition, are 

defined as a separate dimension with no displayed value.
Representation Limitation

Return

to Index

Annotation Structure:

FCF projected tolerance zone defined as separate DIM

20 NIST MBE PMI CTC Model Verification Results

The length of the projected tolerance zone for this feature 

control frame is defined as a separate dimension.

(See related Presentation Limitation)
Representation Limitation

Return

to Index

Annotation Structure:

FCF text defined as separate note

21 NIST MBE PMI CTC Model Verification Results

The text which defines the between-basis for this feature 

control frame is defined as a separate note annotation.
Representation Limitation

Return

to Index

Annotation Structure:

Threaded hole depth DIM not defined

22 NIST MBE PMI CTC Model Verification Results

The depth dimension for each threaded hole, between the top 

(flat) surface and the bottom (circular) edge, is not defined.
Return

to Index

CAD System Representation Limitations for

Annotation Parameters

23 NIST MBE PMI CTC Model Verification Results
Representation Limitation

Return

to Index

Annotation Parameters:

DIM origin not defined

24 NIST MBE PMI CTC Model Verification Results

The origin for this oriented dimension is not defined.

Test Case
Representation Limitation

Return

to Index

Annotation Parameters:

DIM parameter defined with encoded text

25 NIST MBE PMI CTC Model Verification Results

The thread parameters for this threaded hole dimension 

are defined as encoded text and not as named parameters.
Representation Limitation

Return

to Index

Annotation Parameters:

FCF between-basis defined with encoded text

26 NIST MBE PMI CTC Model Verification Results

The between-basis for this feature control frame is defined 

as encoded text and not with named parameters.
Representation Limitation

Return

to Index

Annotation Parameters:

FCF parameter defined with encoded text

27 NIST MBE PMI CTC Model Verification Results

The unequally disposed modifier in this feature control frame 

is defined as a text symbol and not as a named parameter.
Return

to Index

CAD System Representation Limitations for

Annotation Geometry

28 NIST MBE PMI CTC Model Verification Results
Representation Limitation

Return

to Index

Annotation Geometry:

DIM associated with extra face

29 NIST MBE PMI CTC Model Verification Results

This hole diameter dimension is associated with the 

bottom face of each hole and not just the side faces.
Representation Limitation

Return

to Index

Annotation Geometry:

DIM not associated with complete set of faces

30 NIST MBE PMI CTC Model Verification Results

This counterbore depth dimension is not associated 

with both planar faces and all 4 bottom faces.

Complete Representation Incomplete Representation
Representation Limitation

Return

to Index

Annotation Geometry:

DTS associated with extra face

31 NIST MBE PMI CTC Model Verification Results

This datum target symbol is associated with an extra 

(coplanar) face and not just the specified face.
Representation Limitation

Return

to Index

Annotation Geometry:

DTS not associated with face

32 NIST MBE PMI CTC Model Verification Results

This datum target symbol is not associated with the 

face on which the datum target is located.
Representation Limitation

Return

to Index

Annotation Geometry:

DTS not associated with SG point

33 NIST MBE PMI CTC Model Verification Results

This datum target symbol is not associated with the 

supplemental geometry point that defines its 

location on the face.
Representation Limitation

Return

to Index

Annotation Geometry:

FCF associated with extra face

34 NIST MBE PMI CTC Model Verification Results

This feature control frame is associated with an extra 

(coplanar) face and not just the specified face.
Representation Limitation

Return

to Index

Annotation Geometry:

FCF not associated with SG curve

35 NIST MBE PMI CTC Model Verification Results

This feature control frame is not associated with the 

supplemental geometry curve that defines its profile 

direction on this face.
Return

to Index

CAD System Representation Limitations for

Coordinate System Structure

36 NIST MBE PMI CTC Model Verification Results
Representation Limitation

Return

to Index

Coordinate System Structure:

CS not linked to FCF DRF

37 NIST MBE PMI CTC Model Verification Results

The model has no explicit (named) link from each 

feature control frame to the coordinate system that 

represents its datum reference frame.
Return

to Index

CAD System Style Differences for

Annotation Structure

38 NIST MBE PMI CTC Model Verification Results
Style Difference

Return

to Index

Annotation Structure:

DTS requires DFS to be defined

39 NIST MBE PMI CTC Model Verification Results

The system requires a datum feature symbol to be 

defined whenever a datum target symbol is defined.
Style Difference

Return

to Index

Annotation Structure:

FCF requires DFS to be defined

40 NIST MBE PMI CTC Model Verification Results

Because this feature control frame references datum “B”, 

its datum feature symbol must be defined in this saved 

view, although it can be hidden (not visible).

Test Case
Return

to Index

CAD System Style Differences for

Annotation Geometry

41 NIST MBE PMI CTC Model Verification Results
Style Difference

Return

to Index

Annotation Geometry:

DFS edge association is extraneous

42 NIST MBE PMI CTC Model Verification Results

The association of this datum feature symbol with the 

edge of the hole is used to indicate graphical placement. 

It is not specified in the test case.
Style Difference

Return

to Index

Annotation Geometry:

DIM edge association is extraneous

43 NIST MBE PMI CTC Model Verification Results

The association of this dimension with the edge of the 

hole is used to indicate graphical placement. 

It is not specified in the test case.
Return

to Index

CAD System Style Differences for

Supplemental Geometry Structure

44 NIST MBE PMI CTC Model Verification Results
Style Difference

Return

to Index

Supplemental Geometry Structure:

DTS target area is non-solid surface on solid face

45 NIST MBE PMI CTC Model Verification Results

The target area for this datum target is defined as a 

non-solid surface placed on the solid face.
Style Difference

Return

to Index

Supplemental Geometry Structure:

DTS target area is subdivided solid face

46 NIST MBE PMI CTC Model Verification Results

The target area for this datum target is defined as a 

solid face that has been separated from the adjacent 

faces in this solid.
Style Difference

Return

to Index

Supplemental Geometry Structure:

DTS target area is wireframe region on solid face

47 NIST MBE PMI CTC Model Verification Results

The target area for this datum target is defined as a 

wireframe region placed on the solid face.
Style Difference

Return

to Index

Supplemental Geometry Structure:

FCF limited area is non-solid surface on solid face

48 NIST MBE PMI CTC Model Verification Results

The limited area for this feature control frame is 

defined as a non-solid surface placed on the solid face.
Style Difference

Return

to Index

Supplemental Geometry Structure:

FCF limited area is subdivided solid face

49 NIST MBE PMI CTC Model Verification Results

The limited area for this feature control frame is 

defined as a solid face that has been separated from 

the adjacent faces in this solid.
Style Difference

Return

to Index

Supplemental Geometry Structure:

FCF limited area definition inconsistent with target area

50 NIST MBE PMI CTC Model Verification Results

The limited area for this feature control frame is defined as a nonsolid surface placed on the solid face. This is inconsistent with the 

wireframe region used to define datum target areas in this system.
Return

to Index

CAD System Style Differences for

Product Geometry Parameters

51 NIST MBE PMI CTC Model Verification Results
Style Difference

Return

to Index

Product Geometry Parameters:

Threaded hole diameter different than other systems

52 NIST MBE PMI CTC Model Verification Results

The nominal diameter of the geometric hole associated 

with this threaded hole diameter dimension is slightly 

different (by default) than in other systems.
Return

to Index

CAD System Presentation Limitations for

Annotation Visibility

53 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Visibility:

DFS is extraneous when DTS is defined

54 NIST MBE PMI CTC Model Verification Results

This datum feature symbol is not needed when the 

datum target symbol is defined. But the system will not 

allow it to be removed or hidden from this saved view.
Presentation Limitation

Return

to Index

Annotation Visibility:

DFS not visible in specified view

55 NIST MBE PMI CTC Model Verification Results

This model has datum feature symbol “A” defined in 

the first saved view. But it cannot also be displayed in 

the second saved view as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Visibility:

DIM not visible in specified view

56 NIST MBE PMI CTC Model Verification Results

The length of the projected tolerance zone for this feature control frame is 

defined as a separate dimension. (See related Representation Limitation) 

This dimension cannot be displayed in this saved view.
Presentation Limitation

Return

to Index

Annotation Visibility:

DTS visible in wrong view

57 NIST MBE PMI CTC Model Verification Results

These datum target symbols are visible in a default (unspecified) 

saved view which cannot be deleted from the model.
Return

to Index

CAD System Presentation Limitations for

Annotation Layout

58 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Layout:

Counterbore DIM defined as two separate DIM's

59 NIST MBE PMI CTC Model Verification Results

This counterbore dimension cannot 

be defined as a single annotation with 

named parameters that each have 

correct face associations. It must be 

defined as two separate dimensions.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

Countersink DIM defined as two separate DIM's

60 NIST MBE PMI CTC Model Verification Results

This countersink dimension cannot be defined as a single annotation with 

named parameters that each have correct face associations. It must be 

defined as two separate dimensions.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

Threaded hole DIM defined as two separate DIM's

61 NIST MBE PMI CTC Model Verification Results

Test Case

This threaded hole dimension cannot be 

defined as a single annotation with 

named parameters that each have 

correct face associations. It must be 

defined as two separate dimensions.
Presentation Limitation

Return

to Index

Annotation Layout:

DIM limits displayed in reversed order

62 NIST MBE PMI CTC Model Verification Results

The lower and upper limits of this dimension are 

displayed in the reverse order from what is specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

DIM limits not displayed horizontally

63 NIST MBE PMI CTC Model Verification Results

The lower and upper limits of this dimension are not 

displayed horizontally as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

DTS target area diameter defined as separate DIM

64 NIST MBE PMI CTC Model Verification Results

The diameter of this datum target area is defined as a 

separate dimension and not shown in the upper half of 

the datum target symbol as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

FCF text displayed above rather than below

65 NIST MBE PMI CTC Model Verification Results

The “2 SURFACES” text is displayed above this 

feature control frame and not below as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Layout:

FCF text displayed on right rather than below

66 NIST MBE PMI CTC Model Verification Results

The “2 COPLANAR SURFACES” text is displayed on the 

right of this feature control frame and not below as specified.

Test Case
Return

to Index

CAD System Presentation Limitations for

Annotation Location

67 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Location:

DFS not attached to FCF

68 NIST MBE PMI CTC Model Verification Results

This datum feature symbol is not attached to the 

feature control frame as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Location:

DFS overlaps DIM graphics

69 NIST MBE PMI CTC Model Verification Results

The display of this datum feature symbol overlaps the 

dimension to which it is attached.

Test Case
Presentation Limitation

Return

to Index

Annotation Location:

DFS partially buried in solid

70 NIST MBE PMI CTC Model Verification Results

A portion of this datum feature symbol extends into the 

solid model, obscuring its display.
Presentation Limitation

Return

to Index

Annotation Location:

FCF partially buried in solid

71 NIST MBE PMI CTC Model Verification Results

A portion of this feature control frame extends into the 

solid model, obscuring its display.
Return

to Index

CAD System Presentation Limitations for

Annotation Orientation

72 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Orientation:

DIM text orientation is wrong

73 NIST MBE PMI CTC Model Verification Results

This dimension is not oriented horizontally as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Orientation:

DTS text is backwards in this view

74 NIST MBE PMI CTC Model Verification Results

The read direction for the datum target symbols’ text is 

backwards in this view from what is specified.

Test Case
Return

to Index

CAD System Presentation Limitations for

Annotation Lines

75 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Lines:

DFS has no extension line

76 NIST MBE PMI CTC Model Verification Results

This datum feature symbol does not have the specified 

extension line.

Test Case
Return

to Index

CAD System Presentation Limitations for

Annotation Text

77 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Annotation Text:

DIM has extraneous space

78 NIST MBE PMI CTC Model Verification Results

This dimension has an extra space after the pattern 

text (“2X”) which is not specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Text:

DTS text is extraneous

79 NIST MBE PMI CTC Model Verification Results

The target area dimensions shown in the upper half of 

this datum target symbol are not specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Text:

FCF missing note text

80 NIST MBE PMI CTC Model Verification Results

The specified “COPLANAR” text is missing for this 

feature control frame.

Test Case
Presentation Limitation

Return

to Index

Annotation Text:

FCF missing projected tolerance zone length

81 NIST MBE PMI CTC Model Verification Results

The length of the projected tolerance zone for this 

feature control frame is not display as specified.

Test Case
Presentation Limitation

Return

to Index

Annotation Text:

FCF text is extraneous

82 NIST MBE PMI CTC Model Verification Results

The “6X” text above this feature control frame is not specified 

in the test case and is extraneous with the all-around symbol.

Test Case
Presentation Limitation

Return

to Index

Annotation Text:

FCF text is extraneous

83 NIST MBE PMI CTC Model Verification Results

The “(85)” text on the left of this feature control frame 

is not specified in the test case.

Test Case
Return

to Index

CAD System Presentation Limitations for

Coordinate System Visibility

84 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Coordinate System Visibility:

CS visible in wrong view

85 NIST MBE PMI CTC Model Verification Results

The “DEF” and “GHJ” coordinate systems are visible in 

a saved view in which they are not referenced.

Test Case
Return

to Index

CAD System Presentation Limitations for

Coordinate System Name

86 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Coordinate System Name:

CS name not same as DRF

87 NIST MBE PMI CTC Model Verification Results

Because none of the systems enable an explicit link between annotations and 

coordinate systems, the name of each coordinate system should match its 

datum reference frame, thus providing an implicit (visual) link. 

This coordinate system cannot be named with a single letter (“K”).

Test Case
Return

to Index

CAD System Presentation Limitations for

Coordinate System Text

88 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Coordinate System Text:

CS name displayed with extra large text

89 NIST MBE PMI CTC Model Verification Results

The display name for this coordinate system is extremely large.

Test Case
Return

to Index

CAD System Presentation Limitations for

Supplemental Geometry Visibility

90 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Supplemental Geometry Visibility:

SG curve visible in wrong view

91 NIST MBE PMI CTC Model Verification Results

Because the profile of a line feature control frame in this model is 

not specified as visible in this saved view, then its associated 

supplemental geometry curve should not be visible. 

Test Case
Presentation Limitation

Return

to Index

Supplemental Geometry Visibility:

SG point visible in wrong view

92 NIST MBE PMI CTC Model Verification Results

Because the datum targets for this model are not specified 

as visible in this saved view, then their associated 

supplemental geometry points should not be visible. 

Test Case
Return

to Index

CAD System Presentation Limitations for

Saved View Structure

93 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Saved View Structure:

View cannot contain annotations on different planes

94 NIST MBE PMI CTC Model Verification Results

The PMI views in this system are limited to annotations with the 

same view and reading directions. The specified saved view has 

annotations with multiple view and reading directions.

Test Case
Return

to Index

CAD System Presentation Limitations for

Saved View Frustum

95 NIST MBE PMI CTC Model Verification Results
Presentation Limitation

Return

to Index

Saved View Frustum:

View camera position not defined

96 NIST MBE PMI CTC Model Verification Results

Each saved view in the test case has a specified camera position 

(view direction and zoom level). This system is unable to store a 

camera position in its PMI view definition.

Test Case