AmericaMakes.us AmericaMakes.us

Driven by...Driven by...

Approved for Public Release, Unlimited Distribution

Perspectives on Effective AM Data

Management

June 2023

Brandon D. Ribic, PhD.

Technology Director, America Makes

Brandon.Ribic@ncdmm.org
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Overview

The three core activities of the Institute are:

• Develop Additive Manufacturing Technology: 

Projects, Innovation, Technology Transfer, Implementation

• Accelerate Human Capital Development: 

Workforce, Education, Training, Outreach 

• Maintain Collaborative Ecosystem: 

Government, Membership, Community

These focus areas are enabled by: • Membership: Driving engagement and collaboration with our 

nation’s brilliant minds from government, industry and 

academia to advance Additive Manufacturing

• Communications: Driving awareness and spreading the 

word to government, members, stakeholders, community

• Operations: Run by a not-for-profit organization 

with a lean and collaborative structure
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Coordination

• Gaps

• Needs

• Opportunity

Collaboration

• America Makes 

Membership

Catalyzation

• Strategic 

Execution

Technology advancement and collaboration through partnership

 Coordinating known stakeholder needs

• Unique by member type

• Academia, Industry, Gov’t, Non-profit, National 

Labs

 Establishing priorities

• Defining gaps and opportunities

• Milestone, objective, and requirement-based 

approaches

• Leveraging diverse expertise to target benefits and 

impact

 Leveraging collective intelligence/capabilities 

for applied learning

• Execution mechanism

• Many similarities to supply chain development

3

pharmalogisticsiq.com
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

6. AGENDA ITEM 06

Green marketing is a practice whereby companies seek to go above and beyond.

Engagement offers unique opportunity to detail a comprehensive strategy for AM 

industrialization

 Publications, Events, Digital StoreFront

• What we are learning or what is known

• What we can understand better

 Advisory and working groups

• Active learning (knowledge capture, analysis, and sharing)

• Prioritization

 Workshops

• What we need or what we do not have

 Roadmaps

• Organized, prioritized opportunities defined in one place utilizing a common taxonomy

• Four roadmaps – AMSC, EWD, AM Technology, and DoD
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Supply Chain 

& Sourcing

Considerations for Production/Acquisition of AM Products

Design PostProcessing & 

Finishing

Image Courtesy

Quintus 

Technologies

Inspection/ 

NDE

Performance 

& Durability

Cost & Rate

• Capacity

• Capability

• Technical 

Resources 

• Standards and 

Specifications

• Quality Control

• Design data

• Manufacturability

• DfAM

• Data 

Management

• Standards and 

Specifications

• Materials 

Selection

• Tools/Methods

• Mechanical 

properties

• Component/

• Subsystem 

behavior

• Environmental 

Factors

• Standards and 

Specifications

• Maintenance/ 

Repair

• Heat treatment

• Coating

• Machining

• Polishing

• Joining/Brazing

• Standards and 

Specifications

• Reasonable 

criteria

• Inspectability

• Tools/methods

• Standards and 

Specifications

• Complexity

• Cost drivers

• Yield

• Productivity 

drivers
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Qualification and Certification Goals & Objectives

 Gathering and understanding actionable data

• What to look for

• How to measure (tools, techniques, and frequency) accurately and reliably

• What the evidence suggests

• If the evidence coincides with acceptable and unacceptable material or product conditions

• How to react – managing potential for success

 Can be influenced by all steps within the value stream

• Demand and associated productivity

• Feedstock, interim material forms and conditions, and final product form

• Application environment and operating conditions

• Full product lifecycle – production, service, inspection, and maintenance

 Cost and pace to attaining sufficient understanding

• Rigor, tenacity, and wisdom

• Sensible to partner given the technology readiness level in many instances

6
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

6. AGENDA ITEM 06

Green marketing is a practice whereby companies seek to go above and beyond.

Integration of AM within the Larger Manufacturing Industrial 

Base

 Understanding various facets of value stream allow us to respond accordingly 

to our own/customer’s/supplier’s/member’s needs

 The utility and value of AM is conditional

• Application

• Manufacturing Capability

• Performance

• Productivity

• Cost

 The benefits of AM are known, the impact remains to be conditionally verified
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Utility of AM Digital Tools and Analytics are Similarly Conditional

 System of systems

 Communication efficiency

 Connectivity and interoperability of 

pedigreed data

 Collaboration

 Traceability – single source of truth

 Enabling adaptation, agility, and resilience

 Design, processing, quality, and supply 

chain management and optimization

 Accounting for full product life cycle –

design, make, test, service

 Workforce and training

8

Image courtesy of Deloitte

Successful application of digital tools and analytics should deliver agility, resiliency, and cost reduction
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Known Challenges Since 2019/2020

 Resources

• Human and financial

• Knowledge/skills

 Supply Chain Coordination

• Global, all encompassing solutions

• Easing barriers to collaboration

• Facilitation of standardization

 Technology Gaps

• Software

• Hardware

• Methods

 Operations/Best Practices

9

 Must avoid duplication of effort

 Any single entity generates but a 

subset of data

 External data not well characterized, 

or value is unknown

 External data incorporation

 Readily searchable

 Opportunities for advanced analytics

 Securing collaboration and innovation
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

2019 Data Management Workshop Findings

 Dec 2019 Collaboration with ASTM CoE

• Data Management and Schema Workshop

 Outcomes

• Action plan and strategy guide

 Common Data Dictionary

 Common data exchange format

 Automated data acquisition

 Public use cases

 Minimum viable data package

 ASTM Data Subcommittee – ASTM F42.08

 Several standards published and in-process

 Findings and priorities have played an influential role in our actions since then

10
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Standards Prioritization from AMMO AM Workshop 

2021 and 2022

 2021

 Gap QC2: AM Part Classification System 

for Consistent Qualification Standards

 Gap PC4: Machine Qualification 

 Gap D17: Contents of a TDP

 Gap QC1: Harmonization of AM Q&C 

Terminology 

 Gap FMP4: Design Allowables 

 Gap PC2: Machine Calibration and 

Preventative Maintenance 

 Gap FMP1: Material Properties

 Gap PC7: Recycle & Re-Use of Materials 

 Gap PM7: AM Process-Specific Metal 

Powder Specifications

 Gap D15: Design of Test Coupons

11

 2022

 Gap PC4: Machine Qualification 

 Gap QC2: AM Part Classification System for 

Consistent Qualification Standards 

 Gap D17: Contents of an AM TDP 

 Gap PC7: Recycle & Re-use of Materials 

 Gap FMP1: Material Properties

• Gap FMP4: Design Allowables
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Existing AMSC Roadmap v2 Gaps Pertaining to Digital 

Manufacturing

12

Design

 D6: Software-encodable/Machine-readable 

Guidelines

 D8: Machine Input and Capability Report 

 D9: AM Simulation Benchmark Model/Part 

Requirement

 D12: Imaging Consistency

 D13: Image Processing and 2D to 3D 

Conversion

 D17: Contents of a TDP

 D18: New Dimensioning and Tolerancing 

Requirements

 D19: Organization Schema Requirement and 

Design Configuration Control

 D20: Neutral Build File Format

 D22: In-Process Monitoring – Coupling 

predictive analytics (product performance) with 

monitoring data 

 D26: Design for Measurement of AM 

Features/Verifying the Designs of Features 

such as Lattices, etc. 

 D27: Standardized Design for Additive 

Manufacturing (DFAM) Process Chain

Qual and Cert

 QC6: Importing Ultrasound 

Data

 QC7: Protocols for Image 

Accuracy

 Gap QC8: Phantoms 

 Gap QC9: Personnel Training 

for Image Data Set Processing

 Gap QC10: Verification of 3D 

Model 

 Gap QC13: Material Control 

Data and Procedures 

Process Control

 PC1: Digital Format and Digital System 

Control

 PC2: Machine Calibration and 

Preventative Maintenance 

 PC3: Machine Health Monitoring 

 PC4: Machine Qualification 

 PC5: Parameter Control 

 PC6: Adverse Machine Environmental

Conditions: Effect on Component 

Quality

 PC15: Configuration Management: 

Cybersecurity

 Gap PC16: In-Process Monitoring -

registration
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Existing AMSC Roadmap v2 Gaps Pertaining to Digital 

Manufacturing

13

Non-Destructive Evaluation

 NDE2: Standard for the Design and 

Manufacture of Artifacts or Phantoms 

Appropriate for Demonstrating NDE 

Capability. 

 NDE3: Standard Guide for the 

Application of NDE to Objects 

Produced by AM Processes. 

 NDE4: Dimensional Metrology of 

Internal Features. 

 NDE5: Data Fusion

 NDE7: NDE of Counterfeit AM Parts

Maintenance and Repair

 M1: AM Analyses in RCM and CBM. 

 M3: AM Level of Repair Analysis. 

 M5: Model-Based Inspection. 

 M6: Tracking Maintenance. 

 M7: Cybersecurity for Maintenance. 

 M8. Surface Preparation for Additive 

Repair. 
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Objectives for AM Research and Development

14

Design 

Tools/Methods

Finished 

Materials Data

Validated AM 

Processing and 

Post-processing 

Practices

Efficient and 

Relevant Testing 

and Inspection

Standardization

 Stability – repeatable, predictable

 Producibility – scale up to production

 Characterization – as-fabricated properties 

available

 Predictability – validated methods to 

quantify performance

 Supportability – cost-effective inspection 

and repair
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Various Areas of Ongoing Investigation

 Acquisition Methods, Tools, and Protocols

 Design Tools

 Feedstock Characterization

 AM Process KPV’s

 Various AM modalities

 Post-processing – HIP, Heat treatment, 

Finishing

 Inspection – Limits of detectability

 Process Monitoring

 Materials Data (allowables)

 Coatings

 Joining, Part Consolidation, and Integration

 Qualification Methods and Testing
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Characterization of Process Dynamics on Material and 

Product Quality

 Addressing interdependencies of process features on 

material quality

• Physical/mechanical properties

 Key AM process features – outcomes arising due to 

multiple physical processes

 Sources of Microstructural Variation

• Heat flux

• Ambient conditions

• Remelting

• Geometry/ location

 Systematic processing of discontinuities

• Effects of remelting indications

 Accelerate material qualification

• Understanding sensitivities

• Control

• Acceptance criteria

• Process specifications

Laser centerline

Boiling 

temperature

isotherm

xr xf

“Keyhole rear” “Keyhole front”
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Systematic Experimentation to Address Material and Product 

Quality Risks

 Evaluation of key process 

parameters

• Processing location

• Melt sequence

• Laser power

• Travel speed

• Component orientation/geometry

 Influence of heat treatment

• Microstructure

• Mechanical/Physical Properties

 Tensile, Fatigue, Corrosion

 Surface condition

• Coating, machining, as-built

• Corrosion, strength, fatigue

 Standardization

 Training

• Lessons learned

• Key details

M9-

10
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Addressing Operational Risks and Documenting Relevant 

Operational Qualification Considerations 

 Going beyond design point solutions

• “Everything” is an essential variable

 Feature-Based Qualification

• Part family

• Machine learning to predict performance

• Sub-element testing

• Building block approaches

• Demonstrating equivalence

• Quantifying confidence

 Specifications and metrics

• Expansion of process specification protocols

• Commoditizing AM materials/products

 Digital thread approaches

 Addressing Producibility and Supportability

• Yield, Cost, Rate

 Accelerating inspection processes

• Demonstrating alternatives to CT

• Limiting CT through in-situ monitoring

• Data registration – deriving actionable data
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Advanced Sensing and Data Analytics for Evaluating AM 

Material and Product Quality

 Emerging, now-term opportunities

• Measurement to quantify physical 

phenomena

• Enabling comparison of ex-situ 

inspection and in-situ measurement

 Hardware and software development

• Sensing and detection

• Registration

• Correlation – physical processes and 

material properties

• Analysis and Validation

 Advanced analytical techniques

• Artificial Intelligence/Machine Learning

• Statistical analysis - correlation

X-ray CT

-8 -6 -4 -2 0 2 4 6 8

y-position, mm

-8

-6

-4

-2

0

2

4

6

8 Layerwise: Post-Recoat

-8 -6 -4 -2 0 2 4 6 8

x-position, mm

-8

-6

-4

-2

0

2

4

6

8

y-position, mm

Layerwise: Post-Scan

-8 -6 -4 -2 0 2 4 6 8

x-position, mm

-8

-6

-4

-2

0

2

4

6

8

y-position, mm

-8 -6 -4 -2 0 2 4 6 8

x-position, mm

-8

-6

-4

-2

0

2

4

6

8

y-position, mm

Machine Vectors

-8 -6 -4 -2 0 2 4 6 8

x-position, mm

-8

-6

-4

-2

0

2

4

6

8

y-position, mm

Multispectral

-8 -6 -4 -2 0 2 4 6 8

x-position, mm

-8

-6

-4

-2

0

2

4

6

8

y-position, mm

ADR Flaws
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Enabling Expansive Adaptation/Utilization of the 

Technology

 Enabling novel design development with 

relevant, pedigreed materials data

 Reduce barriers for otherwise marginal 

applications

• Development time/cost

 Development of MMPDS Recognized 

Design Allowables

• LPBF of Ti-6Al-4V

• Tensile and fatigue

 Searchable Materials Database

• Data pedigree

• Minimum standard schema

• Accessible and Secure

• Visualization and analysis

 Driving focus on data management 

practices

• Rigor and wisdom
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Project/Program Data Life Cycle

• Facilitated by

• Value Assessment – Goals/Significance

• Tools

• Training

• Methods

• Standards/Specifications/Work 

Instructions

• Security and auditing

• Enduring value derived 

(reuse/interoperable) by

• Planning – structure (pedigree) 

• Transfer

• Storage
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Market Opportunities for AM Product Development Go 

Beyond Aerospace

22

Images courtesy of DARPA and Dreamstime.com

We must consider the utility and impact of AM technologies across various application domains
AmericaMakes.us AmericaMakes.us

Driven by...Driven by...

Approved for Public Release, Unlimited Distribution

Near term efforts and long-term prioritization
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Request for Information

 Methods and Approaches for Sustainable AM Operations

 Announced April 12, 2023

 Responses due May 12, 2023

 Industrialization of AM represents an opportunity to convert demonstrated manufacturing capability to 

capable manufacturing capacity

 There is a need for widespread understanding of manufacturing operational practices that address 

sustainability and robust quality control

 America Makes is looking for inputs from the domestic AM supply chain and the membership to offer 

perspective on needs which address two topic domains:

• Needs and priorities for the development and demonstration of sustainable AM practices and products

• Needs and priorities for the development and demonstration methods and approaches which deliver validated objective 

evidence to substantiate robust AM manufacturing operations (operational qualification)

24
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Operational Qualification

 Borrowed definitions from 2020 AIA publication, “Recommended Guidance for Certification of AM 

Component”

 Operational qualification (OQ) is a myriad of tasks which leads to process control documentation 

substantiated through characterization, analysis, and testing to confirm the process can deliver to 

material specification requirements. 

 OQ commonly occurs between the time the machine is deemed passing site acceptance testing and 

when product qualification starts.

 Quantitatively analyzing AM processes through serial production and over time becomes more 

substantial. 

25
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

OQ may include:

• Monitoring

• Documentation and records keeping

• Powder removal

• Thermal processing - stress relief, HIP, and heat

treatment

• Machining including build plate and support

removal

• Surface finishing/conditioning

26

• Facility controls

• Operator training and qualification

• Work instructions

• Software configuration controls

• KPV controls and documentation

• Machine configuration controls

• Preventative maintenance

• Machine calibration

• Machine requalification

• Feedstock control, management, handling, and reuse

• Build interruptions
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Observations

 The amount, type, and means of sampling, documentation, testing, analysis, etc. can vary depending upon 

the operator, AM process, material, application, and customer. 

 There is a need for establishing consensus balanced with practicality and rigor when it comes to the number, 

practices, types, and methods of testing to foster operational qualification (establishing the amount of 

objective evidence necessary to show an AM process is yielding stable material performance)

 It is worth noting that executing OQ also assumes one has a sufficient understanding of the processes, 

microstructures, and properties which constitute AM material performance

 There is a desire to understand how the supply chain can execute OQ more effectively, not to be confused 

with product acceptance or PQ

 It is believed that with more suppliers understanding the details of effective AM OQ, the time to achieve PQ 

will shorten and we will be able to convert more AM capability to capable AM manufacturing capacity.

27
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Delta Qual

 Qualification of additive manufacturing (AM) machines and materials is a major barrier for the broad adoption 

of AM

 Generating data and models requires significant investment of resources and time in order to produce 

statistically significant data

• Generating test coupons under a controlled process, performing testing, and analysis of the resulting data

 Established processes are “frozen”

• No changes to key process variables allowed

• Currently no standards-based guidance for what is required when change to the process is necessary or desired

 Common route forward for implementing change to a qualified process is total requalification

• Can cost more than $3M per machine/material combination

• Takes several years to complete

 Execute effort which demonstrates the ability to rapidly and affordably update and/or establish a qualified AM 

process that can allow for changes in key AM process, post-processing, and/or material feedstock variables

28
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Important Considerations to Approaches

 Establishing validated feedstock material forms, process controls, and repeatable and 

statistically significant quality (physical and mechanical properties) in a manner that is 

industrially relevant

 Consider factors such as maintenance, calibration, and auditing to substantiate a controlled 

process workflow

 Material and process specifications, control documentation, and other relevant controls will 

be communicated and reviewed by others to establish a valid delta qual is successfully 

demonstrated

 Discussions of approach, data management, and workflow control will occur between teams 

executing novel delta qualifications and other teams executing baseline qualification efforts.

29
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Our Approach to Data Management Remains Firm

 Assessing value proposition of data management and digital based approaches for AM industrialization is critical

• We are actively exploring this question

 Technology development transitioning focus from feasibility to repeatability, transferability, efficiency

• Representative of TRL >4 and driving towards 6/7

 Promoting a culture of data management

• Data management is a cornerstone to our efforts

• Alignment of effort and deliverables to roadmap taxonomy

• KPP’s – success measures

• Transfer of data to membership – America Makes CORE

 Promoting engagement with technical community – consensus building

• Working groups

• Events

• Publications

• Ensuring we identify opportunity to disseminate value, reuse, interoperability of outcomes

 Exploring opportunities to improve organization, storage, and sharing of datasets

30
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Design Offers Expansive Opportunity for Digital Tools and 

Methods

 Product definition alignment to application specific requirements (MBSE) –

safety, performance, manufacturing capability, service life cycle

 Cost savings 

 Accelerating certification by analysis 

 Promoting inspectable complexity

 Light weighting while ensuring rapid qualification and certification

 Accounting for manufacturability limits

 Accounting for process specific thermal history and post-processing

 Multi-physics, multi-objective optimization

31
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Material Characterization, Prediction, Development, and 

Optimization

 Feedstock chemical composition development and qualification

 Virtual testing

 Property prediction and product lifing

 Material process-structure-property relationships

 Tailoring processing conditions to optimize microstructure, quality, and 

performance

 Diminishing risk of process drift or outliers through material desensitization

 Tailoring material characteristics to limit post processing

32
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Process Selection, Development, and Optimization

 Process models or digital twins for achieving productivity, production 

volumes, product scale requirements

 Process control schemes and optimization – material and geometry 

desensitization

 Integration of multiple manufacturing operations and promoting high yield

 Connecting process data to product quality and certification

 Process health monitoring

 Distributed manufacturing

 Connecting manufacturing technology capability with application 

requirements – the right tool for the right job

33
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Value Chain Integration and Verification to Enable Rapid 

Qualification and Certification

 Manufacturing operations/workflow management

 Life cycle analysis tools

 Training, specifications, and interdisciplinary skills

 Common technical data packages

 Cost and value of in-situ process monitoring

 Digitally-based inspection technologies

 Layered solutions for product certification (ex-situ and in-situ) – data 

registration

 Probabilistic tools for product acceptance

 Digital analytics for validating product quality and certification

 Reduced development, qualification, and certification cycles/costs

34
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

Opportunities within the AM Genome for Physics-Based 

Modeling and Predictive Analytics

 Reusable AM data – data management and pedigree are paramount

 Modeling material evolution – process or application operating environment

 Validated predictive modeling (performance, PoD, effect of defects)

 Artificial intelligence and machine learning coupled with physics-based 

understanding

 Trusted approaches to validating models – confidence is necessary for 

adoption

 Model based inspection

 Certification by analysis

35
AmericaMakes.us AmericaMakes.us

Driven by...

Approved for Public Release, Unlimited Distribution

When America Makes

America Works

AmericaMakes.us @AmericaMakes /AmericaMakes