YOUR CENTRAL SOURCE FOR DATA EXCHANGE

www.TranscenData.com

Any mention of commercial products is for information only; it does not imply recommendation or endorsement by NIST.

NIST MBE PMI Validation & Conformance Testing 

CTC-05 Model Translation Validation Results 

June 2015 

Doug Cheney 

CAD Validation Specialist

ITI TranscenData

Doug.Cheney@TranscenData.com 
NIST is Supporting the Discrete Part Manufacturing 

Industry for Long-term Growth

2 NIST MBE PMI CTC Model Translation Validation Results

Manuf.

Industry

PLM 

Software

Vendors

MBE 

Test Data &

Validation 

Testing

High-Quality

MBE 

Software

“Our strategic direction is the 

Model-Based Enterprise (MBE)”

MBE 

Reference 

Data

MBE 

Research

Partnership
NIST MBE PMI Validation and Conformance Testing 

Program Objectives

Develop test cases,

test models and

software algorithms sufficient to 

measure conformance of CAD systems and translators to 

American Society of Mechanical Engineers (ASME) 

standards for Product and Manufacturing Information 

(PMI). 

3 NIST MBE PMI CTC Model Translation Validation Results

ASME Y14.5 Dimensioning and Tolerancing

ASME Y14.41 Digital Product Data Definition Practices 

Web site: http://go.usa.gov/mGVm
Conformance Testing Process

4 NIST MBE PMI CTC Model Translation Validation Results

Test

Cases

CATIA V5

Models

Creo

Models

NX

Models

Verification

SolidWorks

Models

Expert

Review

Validation

STEP 

Models

JT 

Models

PDF

Models

CAx-IF 

(STEP) 

JT-IF

3D 

PDF-IF

Implementor Forum =

Vendor partnership to identify & 

resolve implementation issues
Conformance Testing Terminology

PMI element: An annotation, coordinate system, model property,

supplemental geometry entity, or saved view

Presentation: What the human user sees

(Visual consumption)

Representation: What the downstream software receives

(Automated consumption)

Verification: How well each PMI element is modeled

(CAD system capability)

Validation: How well each PMI element is translated

(Translator capability)

5 NIST MBE PMI CTC Model Translation Validation Results
Selected Combined Test Case (CTC 05) and Models

6 NIST MBE PMI CTC Model Translation Validation Results

CATIA Model

Creo Model NX Model

Test Case
Test Case and Test Model Selection Considerations

Representative PMI elements

– Datums: datum feature symbols and datum target symbols

– Dimensions: basic, bidirectional, statistical, diameter

– Tolerances: multiple geometric types and datum reference frames

– Coordinate system: compound datum reference frame

– Supplemental Geometry: target area and direction curve

– Views: View-specific annotation and supplemental geometry visibility

Representative translations

– 10 – 13 per native format (CATIA V5 R21, Creo 2.0, NX 8.0)

– 11 – 12 per neutral format (STEP AP203/242, JT XT BREP, PDF/PRC)

– 35 total translations

Representative vendor participation

– 3 PLM vendors

– 8 third-party vendors

7 NIST MBE PMI CTC Model Translation Validation Results
Validation Testing Methodology

Each PMI element in a translated model is matched with the 

corresponding element in the corresponding native model.

The representation and presentation data are compared 

between each pair of matched PMI elements.

Each significant difference, that would impeded visual or 

automated consumption, is categorized as a presentation or 

representation validation issue, respectively.

Each issue is grouped by PMI element type and impact 

(removed, changed, or added).

An example of each kind of issue is documented in this 

presentation.

The following slide indexes (underlined) these examples.

8 NIST MBE PMI CTC Model Translation Validation Results
Testing Considerations

SolidWorks was not included as a native format since only 

2 translations (to PDF) were provided by the vendors. 

Each translation was initially validated to determine if all 

element types were included. If not, the vendor was given the 

opportunity to resubmit it (using better configuration options).

A detailed validation report for each translated model, 

highlighting each representation/presentation issue, was 

delivered to the translation vendor.

Many of the translations were produced using pre-release 

translation software.

When an element was added/removed, it was counted as 

both representation and presentation added/removed.

Added percentages were capped at 100%.

9 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Annotation removed

Annotation changed

Annotation added

Coordinate system removed

Coordinate system changed

Coordinate system added

Product geometry removed

Product geometry changed

Product geometry added

Supplemental geometry removed

Supplemental geometry changed

Supplemental geometry added

Model property removed

Model property changed

Model property added

Presentation Issue

Annotation removed

Annotation changed

Annotation added

Coordinate system removed

Coordinate system changed

Coordinate system added

Product geometry removed

Product geometry changed

Product geometry added

Supplemental geometry removed

Supplemental geometry changed

Supplemental geometry added

Saved view removed

Saved view changed

Saved view added

PMI Constructs

Validation Issues (and index to examples)

10 NIST MBE PMI CTC Model Translation Validation Results

If an issue type is not underlined, no 

Test Case Drawings and Models differences were found in this dataset.
Validation Issues NOT Found in This Dataset

Coordinate system representation

– Location or orientation changed

Coordinate system presentation

– Color or name changed

Product geometry representation

– Face removed or added

Product geometry presentation

– Face visibility or color changed

Supplemental geometry representation

– Surface/curve location or orientation changed

– Point location changed

Supplemental geometry presentation

– Surface color changed

Model property representation

– Name or value changed

11 NIST MBE PMI CTC Model Translation Validation Results
Negligible Annotation Differences (Ignored)

Layout (if visible in view)

Text font, size, spacing (if readable)

Leader line routing (if not intersecting any text)

Arrowhead style (if well-formed)

12 NIST MBE PMI CTC Model Translation Validation Results
Translation Capability Measurement Methodology

All translated model entities were subtotaled by element type 

and format.

All validation issues were subtotaled by category, element 

type, impact, and format.

Each issue subtotal was divided by the corresponding 

element subtotal to calculate a percentage.

The max percentage within each group was charted

For example: 11 STEP models contained 242 annotations of 

which 44 had parameter changes (18%). The percentages of 

other kinds of annotation representation changes were lower 

for STEP so 18% is the percentage shown on the chart.

The following slides show these statistics for this dataset.

The names of the native CAD systems are generalized.

13 NIST MBE PMI CTC Model Translation Validation Results
Translated Element Counts by Native and Neutral 

Format

14 NIST MBE PMI CTC Model Translation Validation Results
Annotation 

Validation Summary by Category and Neutral Format

15 NIST MBE PMI CTC Model Translation Validation Results
Coordinate System 

Validation Summary by Category and Neutral Format

16 NIST MBE PMI CTC Model Translation Validation Results
Product Geometry 

Validation Summary by Category and Neutral Format

17 NIST MBE PMI CTC Model Translation Validation Results
Supplemental Geometry

Validation Summary by Category and Neutral Format

18 NIST MBE PMI CTC Model Translation Validation Results
Model Property and Saved View

Validation Summary by Category and Neutral Format

19 NIST MBE PMI CTC Model Translation Validation Results
Summary

Overall

– Every translation had both representation and presentation issues, 

each on more than one type of PMI element

Annotations

– STEP achieved 76% representation, but only 39% presentation

– PDF achieved only 48% representation, but reached 66% presentation

– JT fell in between with 52% representation, 57% presentation and 

about 50% extraneous data added

Coordinate Systems

– JT achieved 83% representation, but only 33% presentation

– STEP achieved only 45% representation, 45% presentation

– PDF achieved only 25% representation, 25% presentation

20 NIST MBE PMI CTC Model Translation Validation Results
Summary (cont.)

Product Geometry

– All formats achieved nearly 100% representation

– STEP and JT achieved 100% presentation

– PDF fell slightly short with 87% presentation

Supplemental Geometry

– All formats achieved between 75% and 89% representation

– JT and PDF achieved about 80% presentation, STEP only 52%

– STEP and PDF added significant extraneous data

Saved Views

– PDF achieved 92% presentation

– JT achieved 58% presentation, but added significant extraneous data

– STEP only achieved 18% presentation

Model Properties

– All formats only achieved between 42% and 56% representation

21 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Annotation removed

Annotation changed

Annotation added

Coordinate system removed

Coordinate system changed

Coordinate system added

Product geometry removed

Product geometry changed

Product geometry added

Supplemental geometry removed

Supplemental geometry changed

Supplemental geometry added

Model property removed

Model property changed

Model property added

Presentation Issue

Annotation removed

Annotation changed

Annotation added

Coordinate system removed

Coordinate system changed

Coordinate system added

Product geometry removed

Product geometry changed

Product geometry added

Supplemental geometry removed

Supplemental geometry changed

Supplemental geometry added

Saved view removed

Saved view changed

Saved view added

PMI Constructs

Validation Issues (and index to examples)

22 NIST MBE PMI CTC Model Translation Validation Results

If an issue type is not underlined, no 

Test Case Drawings and Models differences were found in this dataset.
Return

to Index

Translator Representation Issues for

Annotation Removed

23 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Annotation Removed

24 NIST MBE PMI CTC Model Translation Validation Results

The definition of this datum target symbol was lost.

Native Model Neutral Model
Return

to Index

Translator Representation Issues for

Annotation Changed

25 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Annotation Edge Association Changed

26 NIST MBE PMI CTC Model Translation Validation Results

The associativity of this straightness tolerance with its 

represented line element was lost.

Native Model Neutral Model
Representation Issue

Return

to Index

Annotation Face Association Changed

27 NIST MBE PMI CTC Model Translation Validation Results

The associativity of this dimension with its faces was lost.

Native Model Neutral Model
Representation Issue

Return

to Index

Annotation Face Association Changed

28 NIST MBE PMI CTC Model Translation Validation Results

Part of the associativity of this feature control frame with its 

faces was lost.

Native Model Neutral Model
Representation Issue

Return

to Index

Annotation Parameter Changed

29 NIST MBE PMI CTC Model Translation Validation Results

While the displayed text for this dimension is correct in the 

neutral model, the semantic (internal) value has changed.

Native Model Neutral Model
Return

to Index

Translator Representation Issues for

Annotation Added

30 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Annotation Added

31 NIST MBE PMI CTC Model Translation Validation Results

These “sketch” dimensions are defined in the native model 

but not visible in the MBD views. They were translated into 

the neutral model and made visible. 

Neutral Model
Return

to Index

Translator Representation Issues for

Coordinate System Removed

32 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Coordinate System Removed

33 NIST MBE PMI CTC Model Translation Validation Results

Native Model Neutral Model

Coordinate system “A-B” is defined in the native model but 

not in the neutral model.
Return

to Index

Translator Representation Issues for

Product Geometry Changed

34 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Face Geometry Changed

35 NIST MBE PMI CTC Model Translation Validation Results

This fillet surface is not tessellated correctly in the neutral 

model (which intentionally only contains graphical geometry).

Native Model Neutral Model
Representation Issue

Return

to Index

Face Topology Changed

36 NIST MBE PMI CTC Model Translation Validation Results

The native model is defined as a closed solid. The geometry 

in the neutral model was translated into an open shell.

Native Model Neutral Model
Return

to Index

Translator Representation Issues for

Supplemental Geometry Removed

37 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Curve Removed

38 NIST MBE PMI CTC Model Translation Validation Results

The boundary of this target area is defined with wireframe 

curves in the native model that were not translated into the 

neutral model.

Native Model Neutral Model
Representation Issue

Return

to Index

The origin point for the coordinate system of this model was 

not translated.

Point Removed

39 NIST MBE PMI CTC Model Translation Validation Results

Native Model Neutral Model
Representation Issue

Return

to Index

Surface Removed

40 NIST MBE PMI CTC Model Translation Validation Results

This target area is defined by a non-solid surface that was 

not translated into the neutral model.

Native Model Neutral Model
Return

to Index

Translator Representation Issues for

Supplemental Geometry Added

41 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Curve Added

42 NIST MBE PMI CTC Model Translation Validation Results

These “sketch” curves are defined in the native model but not visible in the 

MBD views. They were translated into the neutral model and made visible.

Native Model Neutral Model
Representation Issue

Return

to Index

Several extraneous points were added to this model.

Point Added

43 NIST MBE PMI CTC Model Translation Validation Results

Native Model Neutral Model
Representation Issue

Return

to Index

Surface Added

44 NIST MBE PMI CTC Model Translation Validation Results

A non-solid surface, that overlays this face and its logo, was 

added into the translated model. 

Native Model Neutral Model
Return

to Index

Translator Representation Issues for

Model Property Removed

45 NIST MBE PMI CTC Model Translation Validation Results
Representation Issue

Return

to Index

Model Property Removed

46 NIST MBE PMI CTC Model Translation Validation Results

These model properties (parameters) are defined in the 

native model but not in the neutral model.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Annotation Removed

47 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Annotation hidden

48 NIST MBE PMI CTC Model Translation Validation Results

The highlighted annotations were translated but not 

displayed in this view.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Annotation Changed

49 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Annotation Color Changed

50 NIST MBE PMI CTC Model Translation Validation Results

The color of these annotations was changed from blue to 

black.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Lines Changed

51 NIST MBE PMI CTC Model Translation Validation Results

The divider line for this datum target symbol was moved to 

the location indicated by the cursor (right view).

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Location Changed

52 NIST MBE PMI CTC Model Translation Validation Results

This datum feature symbol was moved so that it is partially 

buried in the solid.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Name Changed

53 NIST MBE PMI CTC Model Translation Validation Results

The display names of five annotations were changed. 

The two highlighted annotations were changed to the same name.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Orientation Changed

54 NIST MBE PMI CTC Model Translation Validation Results

The orientation of this dimension was reversed in this view.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Text Changed

55 NIST MBE PMI CTC Model Translation Validation Results

The symbol for this feature control frame was changed.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Text Changed

56 NIST MBE PMI CTC Model Translation Validation Results

The statistical tolerance symbol for this dimension was lost.

Native Model Neutral Model
Presentation Issue

Return

to Index

Annotation Text Changed

57 NIST MBE PMI CTC Model Translation Validation Results

The trailing zeros for these basic dimensions were lost.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Annotation Added

58 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Annotation Shown

59 NIST MBE PMI CTC Model Translation Validation Results

This datum target symbol was added to the display of this 

view.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Coordinate System Removed

60 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Coordinate System Hidden

61 NIST MBE PMI CTC Model Translation Validation Results

Coordinate system “A-B” is defined in the neutral model but 

not displayed when this view is activated.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Product Geometry Changed

62 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Face Tessellation Inaccuracy

63 NIST MBE PMI CTC Model Translation Validation Results

The indicated tessellation vertices have gaps up to .009 inch 

off of the precise geometry.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Supplemental Geometry Removed

64 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Curve Hidden

65 NIST MBE PMI CTC Model Translation Validation Results

The represented line element for this straightness tolerance 

was translated but removed from this view.

Native Model Neutral Model
Presentation Issue

Return

to Index

Point Hidden

66 NIST MBE PMI CTC Model Translation Validation Results

The origin point for the coordinate system of this model was 

removed from this view.

Native Model Neutral Model
Presentation Issue

Return

to Index

Surface Hidden

67 NIST MBE PMI CTC Model Translation Validation Results

The non-solid surface representing the target area for this 

datum target was removed from this view.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Supplemental Geometry Changed

68 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Curve Color Changed

69 NIST MBE PMI CTC Model Translation Validation Results

The color of the represented line element for this straightness 

tolerance changed from light to dark blue.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Supplemental Geometry Added

70 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

Curve Shown

71 NIST MBE PMI CTC Model Translation Validation Results

This curve was added to this view in which it is not used.

Native Model Neutral Model
Presentation Issue

Return

to Index

Point Shown

72 NIST MBE PMI CTC Model Translation Validation Results

Several extraneous points were added to this view.

Native Model Neutral Model
Presentation Issue

Return

to Index

Surface Shown

73 NIST MBE PMI CTC Model Translation Validation Results

This non-solid surface was added to this view in which it is 

not used.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Saved View Removed

74 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

View Removed

75 NIST MBE PMI CTC Model Translation Validation Results

The saved view definitions were lost completely.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Saved View Changed

76 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

View Frustum Changed

77 NIST MBE PMI CTC Model Translation Validation Results

When this view is activated in the neutral model, the camera 

position, orientation and zoom level is different.

Native Model Neutral Model
Presentation Issue

Return

to Index

View Name Changed

78 NIST MBE PMI CTC Model Translation Validation Results

The definition of this view was translated incompletely, 

including the loss of its display name.

Native Model Neutral Model
Return

to Index

Translator Presentation Issues for

Saved View Added

79 NIST MBE PMI CTC Model Translation Validation Results
Presentation Issue

Return

to Index

View Added

80 NIST MBE PMI CTC Model Translation Validation Results

Several views were added which contain partial definitions of 

the views in the native model.

Native Model Neutral Model
Return

to Index

PMI Constructs

81 NIST MBE PMI CTC Model Translation Validation Results
PMI Constructs in CTC 05

82 NIST MBE PMI CTC Model Translation Validation Results

Return

to Index
Return

to Index

Test Case Drawings and Models

83 NIST MBE PMI CTC Model Translation Validation Results
Combined Test Case 5, Saved View MBD_A Drawing

84 NIST MBE PMI CTC Model Translation Validation Results

Return

to Index
Combined Test Case 5, Saved View MBD_A Models

85 NIST MBE PMI CTC Model Translation Validation Results

Return

to Index
Combined Test Case 5, Saved View MBD_B Drawing

86 NIST MBE PMI CTC Model Translation Validation Results

Return

to Index
Combined Test Case 5, Saved View MBD_B Models

87 NIST MBE PMI CTC Model Translation Validation Results

Return

to Index