Digital Transformation in Steel Inspection
Toshiba MetalSpector Customer Case Study
20211118

Authors:
Hiroshi Yamamoto
CTO, Toshiba Digital Innovation Technology Centerhiroshi78.yamamoto@toshiba.co.jp
Daniel Young
Senior Manager, Toshiba America, Inc.
Daniel1.young@toshiba.com

CONTENTS
1 Overview .......................................................................................................................... 3
1.1
1.2

Introduction ........................................................................................................................ 3
Digital Transformation Background ...................................................................................... 3

2 Metallurgical Inspection Background ................................................................................. 4
2.1

Steel Industry Metallurgical Inspection Process .................................................................... 4

3 Customer Analysis............................................................................................................. 6
3.1

Key Observations................................................................................................................. 6

4 System Development ........................................................................................................ 8
4.1

Updated Steel Inspection System ......................................................................................... 8

5 Digital Transformation .................................................................................................... 10
5.1
5.2
5.3

New Service Offering ......................................................................................................... 10
Technology Enablers .......................................................................................................... 11
Transformation Achieved................................................................................................... 12

6 Summary ........................................................................................................................ 13
7 Acknowledgements......................................................................................................... 14

FIGURES
Figure 2-1 Metallurgical inspection process for grading steel samples ........................................................ 4
Figure 2-2 Non-metallic inclusion measurement system .............................................................................. 5
Figure 4-1 Updated steel inspection analysis ................................................................................................ 8
Figure 5-1 Steel inspection cloud services ...................................................................................................10

2

November 2021

Digital Transformation in Steel Inspection

1 OVERVIEW
1.1

INTRODUCTION

Toshiba has been selling MetalSpector, a leading-edge steel inspection equipment, to several
major steel companies for several decades1. Recently, as part of Toshiba’s Digital Transformation
strategy, Toshiba redesigned MetalSpector into a steel grading evaluation service, intended toreduce steel inspector’s workloads while assuring the data integrity of the steel grading process.
The result of this redesign uncovered a new set of services focused on how the inspection
equipment is used by the customer, rather than focusing solely on improvements to how theinspection equipment works.

1.2

DIGITAL TRANSFORMATION BACKGROUND

Toshiba is currently transforming from a hardware-sales business model to a subscription service
model for Infrastructure Services, which is the core segment of Toshiba’s business2. Whilecontinuing to sell hardware and infrastructure systems to our customers, we are shifting towards
offering O&M (Operation and Maintenance) services as a subscription. Our immediate focus of
O&M services is for Toshiba hardware, and then extending this scope to competitor hardware in
the future. This is a major reason why Toshiba is prioritizing connectivity using industry standards.
The importance of this transformation is realized by a set of new managed services that are
directly connected to customer value. For customers, these managed services are enabled bytechnology enablers such as machine learning, cloud computing, and data management tools;
with these enablers, the services can adapt to evolving needs, improve the more they are used,
and be updated without incurring downtime.

1

Toshiba: Applying AI Image Processing Services to Steel Product Quality Management. [Online] 2019.
https://www.global.toshiba/ww/company/digitalsolution/articles/tsoul/31/004.html.
2 Toshiba: Toshiba to Reorganize Headquarters Corporate Staff toward Realization of Toshiba Next Plan. [Online]
Toshiba, 2019. https://www.global.toshiba/ww/news/corporate/2019/01/pr3101.html.

IIC Journal of Innovation

3

2 METALLURGICAL INSPECTION BACKGROUND
2.1

STEEL INDUSTRY METALLURGICAL INSPECTION PROCESSIn the Quality Assurance (QA) department of a steel manufacturer, samples of the outgoing steel
are extracted, and metallography is applied to assure product quality by QA inspectors using a
microscope. The inspectors perform the measurement for its grade judgment based on steel
industry standards and company quality requirements3 4.

Figure 2-1 Metallurgical inspection process for grading steel samples

The non-metallic inclusion measurement system automatically measures non-metallic inclusions
in steel samples by using an auto-focus optical microscope, X-Y stage, color area sensor camera,
and an image processor. The sample holder with the test piece set is moved to a predetermined
position on the X-Y stage, and image input is performed.

3

ASTM: E-45, Standard Test Methods for Determining the Inclusion Content of Steel. [Online] 2018.
https://www.astm.org/Standards/E45.htm.
4 JIS. JIS G-0555, Microscopic testing method for the non-metallic inclusions in steel. [Online] 2020.
https://webdesk.jsa.or.jp/books/W11M0090/index/?bunsyo_id=JIS+G+0555%3A2020.

4

November 2021

Digital Transformation in Steel Inspection

Figure 2-2 Non-metallic inclusion measurement system

Steel grade measurements take time to perform, depending on the number of inclusions found,on the order of minutes; this is because the measurements are in the sub-micrometer range andeach inclusion must be measured. With this time scale in mind, decreasing the measurement
time by even one minute is a substantial amount. Even with the most modern algorithms andhardware, semi-automated measurements using ASTM or JIS point calculation methods can takebetween 2-6 minutes to perform5. The measurement times can be lowered with bettermeasurement algorithms and autofocusing functions.
While the measurement system can help identify non-metallic inclusions, it is the inspector who
must make the final evaluation of the samples and grade them according to regional regulations,international standards, and customer proprietary measurement methods 6 7. This assessment
requires considerable skill and training, so the steel manufacturer highly values their skilledinspectors.

5

Toshiba: Applying AI Image Processing Services to Steel Product Quality Management. [Online] 2019.
https://www.global.toshiba/ww/company/digitalsolution/articles/tsoul/31/004.html.

6 ASTM. ASTM E-45, Standard Test Methods for Determining the Inclusion Content of Steel. [Online] 2018.

https://www.astm.org/Standards/E45.htm.
7

JIS: JIS G-0555, Microscopic testing method for the non-metallic inclusions in steel. [Online] 2020.
https://webdesk.jsa.or.jp/books/W11M0090/index/?bunsyo_id=JIS+G+0555%3A2020.

IIC Journal of Innovation5

3 CUSTOMER ANALYSIS
With increases in steel production come increases in steel quality samples, which in turns adds
to the demand on inspectors (ex: longer hours and/or more inspectors). Thus, several issues
arise from the existing inspection process:Steel Inspection Process Issues
Labor Intensive QA
Process

Because the grading process requires human inspectors to grade eachsample, the grading process can quickly become the largest bottleneck in the
manufacturing process, especially when the number of veteran inspectors is
low.

Long Inspection Hours

Steel inspection is sampled per the amount of steel produced and not by a
set number of samples per day. As more steel is produced, more samples
are required to be graded, which requires longer hours for inspectors.

Lack of Consistency

As inspectors are asked to work longer hours, they can become fatigued, andit becomes increasingly difficult to maintain a level of consistency in thegrading process. Increasing the number of inspectors can also create a lackof consistency from one inspector to another.

Product Quality Data
Fraud

3.1

As the demand for more output increases, there can be pressure to falsify
records and report gradings that are not correct. In recent years, it has
become public that several steel manufacturers around the world havefalsified their quality records, resulting in large regulatory fines, litigation,and damage to company reputation.

KEY OBSERVATIONS

While the inspection measurement itself is not a repetitive task, much of the initial setup work is(i.e., preparing the samples for measurement); therefore, automation can be applied to some(but not all) of the inspection processes. In the past, improvements to image acquisition andprocessing relied heavily on hardware improvements, so it was sensible to offer equipment
upgrades. Today, however, it is not the local hardware that provides the largest improvements
in image analysis; rather, it is the advancements in image processing algorithms that reduceanalysis time and improve accuracy. Therefore, the idea of continuous improvements, offeredas-a-service, provide greater benefits to the customer without incurring the negative impacts ofhardware upgrades (i.e., the associated downtime from upgrading equipment).
As standards develop in the areas of automated inclusion measuring, the system should bedesigned to take further advantage of new measurement algorithms, autofocus functions, and6

November 2021

Digital Transformation in Steel Inspection
new statistical methods (also called “clean steel measurements”) 8. By leveraging the vast
amounts of metallographic structure inspection data along with these image processing
technologies, advanced deep learning techniques can be developed to inspect the image samples
and detect non-metallic inclusions. With experienced human inspectors to help train these AImodels, productivity should increase, so long as the inspector is able train the models withrelative ease and not be burdened with understanding a new set of tools to perform the training.
Currently, national regulations require human inspection and prohibit the use of AI to perform
inspection by itself. As the technology matures, and AI models are proven to be as reliable asexperience human inspectors, such regulations could change. In fact, the steel industry has
begun developing standards for automatic inclusion inspections 9. However, for the present, it is
not expected to eliminate human inspectors; rather, human inspectors are considered high value
assets and critical to the measurement process.

8

Toshiba: Applying AI Image Processing Services to Steel Product Quality Management. [Online] 2019.
https://www.global.toshiba/ww/company/digitalsolution/articles/tsoul/31/004.html.
9 ASTM: ASTM E768 - 99(2018), Standard Guide for Preparing and Evaluating Specimens for Automatic Inclusion

Assessment of Steel. s.l. : ASTM, 2018.

IIC Journal of Innovation

7

4 SYSTEM DEVELOPMENT
The steel manufacturer requested improvements to the existing equipment to overcome theirprocess issues. The key objectives were to a) reduce inspector workloads, and b) provide
automatic auditing capabilities. These objectives would address the issues with higher demandfor inspections and longer working hours, as well as the serious problem of false record keeping.

4.1

UPDATED STEEL INSPECTION SYSTEM

Figure 4-1 Updated steel inspection analysis

The new system (Figure 4-1) was designed with an IIoT architecture, using a combination of Edge,
Platform, and Enterprise services in a 3-Tier Architecture Pattern, and implemented using
containerized micro-services10 11.
The system process flow was developed as follows:
1.

2.

Transfer steel sample’s image data from microscope to image historian database• By using the same equipment as before, the original process remains very similar
for the inspector.
Transfer image data with sequence number to cloud for feeding an AI model estimator

10

Toshiba: Applying AI Image Processing Services to Steel Product Quality Management. [Online] 2019.
https://www.global.toshiba/company/digitalsolution/articles/tsoul/31/004.html.
11 Toshiba: Toshiba Internet Reference Architecture. [Online] 2018.
https://www.global.toshiba/company/digitalsolution/articles/tsoul/32.html.

8November 2021

Digital Transformation in Steel Inspection
•
3.

4.

By automatically recording image samples, an audit trail can be established that
does not require additional workload for the inspector.
Store steel grade estimate from AI model and sequence number into log management
database• Incorporation of AI estimation can improve the speed and efficiency of the grading
process by recognizing routine inclusion patterns without inspector intervention.
In addition, the audit trail is further maintained by pairing the AI analysis with theoriginal image sample.
Inspectors determine final quality decision by referring to estimated steel grade• By automating the image recording and AI estimation process, the inspector canbetter concentrate on the steel grading process. The AI estimations aids theinspector by identifying routine inclusion patterns, allowing the inspector moretime to evaluate difficult sample images.

For the auditing and steel grade evaluation services, it was important to have permission toaccess to the steel image sample data. The image samples must be sequenced and stored forthe auditing service, and they must be and analyzed by the AI model to perform a grading
estimate. Because these operations occur outside the customer’s environment, permission must
be granted by the customer to allow these operations to be performed.
Because of the existing relationship with the steel manufacturer, including previous operational
improvements, a high level of trust had already been established. Moreover, it was easy for themto realize the value of the auditing and AI-based evaluation services; so, allowing access to thesample data was not a concern. Nevertheless, it is always important to be aware of data
ownership rules, understanding both business and legal concerns, regardless of the customer’slevel of trust in the system or the supplier.

IIC Journal of Innovation

9

Digital Transformation in Steel Inspection

5 DIGITAL TRANSFORMATIONThe redesign of the steel inspection system was initially focused on improving how the system
worked. This is a typical outcome for a company that is focused on hardware sales. However, as
part of the company’s new focus on O&M services, the customer environment was also observed
as a way of understanding how the QA inspectors used the steel inspection equipment.

5.1

NEW SERVICE OFFERING

In partnering with the steel manufacturer and closely observing their process and product quality
inspection, we were able to highlight a new process issue:(New) Steel Inspection Process Issues

Skills Transfer

As demand increases, there is also a demand for more inspectors to reducethe burden on existing inspectors. Because training is passed on fromveteran inspector to junior inspector, manufacturers must balance the needto train new inspectors versus the more immediate need to grade steel
samples. Inspector training can take years to acquire a comparable set ofskills.

As a result, we developed a new training service, which was reviewed and approved by thecustomer. This new service takes the AI inference model for steel grade determination and uses
it as a learning model to teach junior inspectors.

Figure 5-1 Steel inspection cloud services

10

November 2021

Digital Transformation in Steel Inspection

As a grade determination service, that AI inference model develops its own analysis skills,
automatically, with each steel sample it analyzes. It also adapts to the analysis from veteran
human inspectors, who have the final judgement decision. As the human inspector performs
their normal measurement analysis, they are automatically training the AI model, with noadditional burden on the inspector.
As a training service, the AI model can provide real-world image samples to junior inspectors andteach them to measure for non-metallic inclusions. Since the veteran inspectors continually train
the AI model, they are also providing new training material for the junior inspectors. Thus, theburden on veteran inspectors is reduced, and junior inspectors gain experience from real-world
data samples.
The value of this new service offering was immediately recognized by the customer, as it reduced
the number of veteran inspectors needed for training purposes, while simultaneously increasing
the skill levels of junior inspectors.

5.2

TECHNOLOGY ENABLERS

The key technology enablers for the metal inspection services were as follows:
Company Perspective

Artificial Intelligence

The Steel AI Image Processing Service can analyze steel samples,
automatically determine non-metallic inclusions, and estimate grades ofsteel grain sizes. The AI Model can also be trained by inspectors, continually
improving the model.

Cloud Computing

The Steel AI Image Processing Service runs in the cloud, so there is no needto add additional hardware on-site. The service is therefore not bound by
local processing power, and updates are automatically applied with nodowntime. In the case of an Internet outage, the overall system can still
function without the AI Service, providing fault tolerance.

Image Recognition

The image recognition processing algorithms are applied at the imaging
station (i.e., at the Edge) and are compared for speed against standard steel
inspection methods (ASTM and JIS), and for accuracy against visual inspection
from an electron microscope.

Data Management

All image data for the steel grading is securely stored and managed to ensure
the data’s integrity and consistency. The data management system allows any
inspectors to read the data, but not update the data.

IIC Journal of Innovation

11

Digital Transformation in Steel Inspection

5.3

TRANSFORMATION ACHIEVED
Company Perspective
Mindset Focus

Changed from Product Function oriented to Product Usage oriented

Revenue Stream

Changed from NRE and fixed purchase to As-A-Service

Solution Focus

Changed from performance-focused, hardware-oriented solutions to userfocused products and services

Business Opportunities Changed from highly customized integrations to modular services that can beadapted to specific customer needs

From Toshiba’s perspective, the use of Data Management techniques, AI for image recognition,
and Cloud Computing resources allowed the key product functional concerns to be addressed.
By proving the value of the AI image estimation and data auditing, the customer became
receptive to providing customer data as part of a subscription contract. This in turn allowed a
new set of services, based on training the customer’s inspectors, to be realized.
Because the AI imaging service found success in a real word scenario, the value of similar business
opportunities was recognized. Image inspection is highly relevant to other manufacturing
applications, and new image inspection services can be provided to customers with similar needs.
Customer Perspective
Data Integrity

Workload Efficiency

The data integrity is assured by storing in a secure database which eliminates
data fraud.
Inspectors can focus on the more difficult inspection samples and train
junior inspectors.

Continuous
Improvement

The AI Model can be improved over time, providing continuous improvement.

Skills Transfer andTraining

Once the AI model training is complete, it is used for not just inspection, butalso used for junior inspector education, which enables improved transfer ofskills to the next generation

From the customer’s perspective, the key objectives of productivity and data integrity were fully
satisfied. Inspectors were able to focus more time on difficult inspections as well as training forjunior inspectors. The AI modeling service can be trained over time to adapt to different types
12

November 2021

Digital Transformation in Steel Inspectionof steel samples, and newer methods of inspection. Moreover, the AI modeling estimations wereshown to be useful for inspector training, providing a completely new training service to thecompany that further alleviates the burdens on veteran inspectors.

6 SUMMARY
From a hardware-sales business model perspective, the redesigned system solves the customer’s
pain points via automation, with a strategy of minimizing the human role in the process.
However, this is not the customer’s objective, as human inspectors are a critical resource andhighly valued. The customer’s objective is in fact to better enable inspectors to perform theircritical role of quality assurance.
Gartner describes digital humanism as a means of redefining the way people’s goals can beachieved. Contrasted with automation, digital humanism focuses on minimizing the complexities
and difficulties a human user has when using a system. Instead of automating system processes
and tasks for the purpose of eliminating human operation, the goal becomes instead to empowerhuman users to do more with the system, potentially unveiling new uses and providing new valuethat was otherwise unachievable 12.
It is important to note that the AI modeling is not intended to eliminate the human inspector
from the process. In fact, it is critical to retain skilled inspectors for the steel manufacturers
competitive advantage. Regulations also prevent the sole use of AI to perform metallic inclusion
inspection, so until such regulations are updated, human inspection is necessary. The use of AI
modelling is therefore intended to offload many mundane tasks and allow the inspectors to focus
on the non-repetitive analysis steps.
While automation is a key process improvement, it is not the end goal. Supporting the steel
inspector’s ability to identify issues with the steel production process is the end goal; this involves
both machine automation as well as improved human involvement. By combining automation
with digital humanism, the focus on providing customer value can be maintained 13.
The successful deployment of MetalSpector helped validate the managed services approach at
the business level. Since then, new image inspection services for other manufacturing andinfrastructure applications have been developed, opening new business opportunities. For
example, Toshiba has created a new managed service for the pharmaceutical industry, tomeasure surface defects in PTP packaging sheets. Similar managed services are also being
developed for the inspection of film materials, paper, and non-woven cloth. Because these use

12 Brian Prentice: Digital Humanism Is a Key to Digital Success. s.l. : Gartner, 2021.
13 IIC; Industrial Internet Reference Architecture IIRA 1.9. [Online] 2019. https://www.iiconsortium.org/pdf/IIRA-

v1.9.pdf

IIC Journal of Innovation

13

Digital Transformation in Steel Inspection
cases also involve sample inspection, many of the image analysis components developed for
MetalSpector can be applied to these inspection cases, too.
The migration from hardware sales to As-A-Service sales is a challenging effort for many
companies. Our fundamental change was from a reactive product development design toproactive service offerings. Digital Transformation Enablers, such as AI, allow continuous
improvement to the customer’s operations, without additional knowledge and effort on theirpart. It also helps uncovers new use cases that help solve additional problems a customer has.
This can lead to the creation of new services, providing new value to customers.

7 ACKNOWLEDGEMENTS
The views expressed in the IIC Journal of Innovation are the author’s views and do not necessarily
represent the views of their respective employers nor those of the Industry IoT Consortium®.
© 2021 The Industry IoT Consortium® logo is a registered trademark of Object Management
Group®. Other logos, products and company names referenced in this publication are propertyof their respective companies.

➢

Return to IIC Journal of Innovation landing page for more articles and past editions

14

November 2021