Non-Desk work DX with Connected Worker: Attracting
attention around the world in the new COVID-19 era
Jun Kuchii

Chief Operating Officer/ Futurist
Fairy Devices Inc.

1. What is Connected Worker and how is it
advancing Non-Desk work DX?

Even though the level of skill required and workload areincreasing, most manufacturers and infrastructure industries
continue to face the challenge of workforce shortages. On theother hand, automation using robotics is not progressing fastenough. Robots are skilled at handling fixed tasks, but they cannot
work with the flexibility that people can. For some tasks, such as
bulk materials handling that people can do easily, even as a parttime job, most attempts to implement them with robots have notbeen practical.
Robots are also expensive, so it is difficult for all but thelargest companies, such as automobile manufacturers, to adopt
them. Under these conditions, digital technologies that are able
to reduce the workload for on-site workers and multiply the added
value created by several times have begun to attract attention.
An example of such technology, which uses digital devicesthat on-site workers can wear to receive AI and IT support,improve their skills and implement DX, is a solution that we call“Connected Worker.”
Connected Worker can improve the standard of on-site work
to that of an experienced technician through use of various digital
supports including: (1) remote support from an experienced
technician, (2) automatic collection of data during work (automatic
creation of reports, etc.), (3) collection of work-site data, generationof hazard alerts, (4) training and manual updates based on pastwork data, and (5) work support using data and AI.
Connected Worker is starting to be introduced to achievecomplex goals such as “improving work productivity,” “managingworker health and safety,” and “preventing work errors andomissions,” for on-site work that is intolerant of error, such as inhigh-tech materials facilities, chemical plants, oil refineries, andelectrical power plants.

2. Non-Desk work DX achievable with
Connected Worker

Currently, particular benefits from implementing Connected
Worker are: more efficient business travel, because the onsite worker can receive remote help working together with anexperienced technician on technical tasks; and more efficientcompletion of non-core tasks during daily work, such as decisionmaking, creation of reports, and collection of evidence and otheron-site data.
I expect many readers have had the difficult experience, asI have, of introducing DX or AI, such as information in real
documents that are not converted to digital data, or huge amountsof data which require a tremendous workload to cleanse.
Introducing Connected Worker enables you to improvebusiness travel and non-core work efficiencies, while alsocollecting data.
■ Figure 2: Connected Worker use cases and effects■ Figure 1: Value provided by Connected Worker

Copyright © 2021 Fairy Devices Inc. All rights reserved.

Figures 3, 4 and 5 show use cases of DX in on-site operation ofa factory, a construction site and for maintenance, respectively. We
have found that it can also be effective in logistics, warehousing
and transport domains. For example, by assigning a speech
recognition digital device to long-distance truck drivers, they caninteract with the fleet management system while driving, within
the scope of what is permitted legally.
Copyright © 2021 Fairy Devices Inc. All rights reserved.

New Breeze Summer 20219

■ Figure 3: Connected Worker use case in a factory

Copyright © 2021 Fairy Devices Inc. All rights reserved.

■ Figure 4: Connected Worker use case on a construction
site

Copyright © 2021 Fairy Devices Inc. All rights reserved.

■ Figure 5: Connected Worker use case for maintenanceshortages. In the 2020s in particular, more skilled technicians areretiring each year, and the shortage of personnel is expected tomake it increasingly difficult to maintain operations in factories
and other infrastructure. This is one of the major reasons why DX
is needed in the workplace.
A shortage of workers is emerging in over 90% of enterprises
in manufacturing industries and for more than 30%, this has
already affected their business＊1. In infrastructure industries
requiring qualified technicians, such as electrical power, a shortageof electrical safety workforce numbering in the thousands isexpected by around 2030＊2.
On the other hand, factories and infrastructure are getting
more complex and advanced, so it is expected that technologists
working in the field will require even more advanced judgment
and knowledge in the future than they do now.
In other words, to maintain its GDP and highly developed
infrastructure, Japan will need to increase both the quality and thenumber of technologists in the workforce.
The government has advocated a “Connected Industries,”
approach to address this situation, promoting DX in all industries.
IoT and Digitalization have been promoted for several years,
and Connected Industries expands these ideas, using digitaltechnologies to create better working environments, to pass ontechniques, skills and knowledge, to increase productivity andeven to try to solve societal issues＊3.
■ Figure 6: E xamples of the benefits of ConnectedIndustries(*3 from P19 of the METI document)

Copyright © 2021 Fairy Devices Inc. All rights reserved.

* Refer to the following for use case details.
https://fairydevices.jp/cws

3. Why is Digital Transformation (DX) neededin the first place?

I admit that for most readers I am preaching to the choir,
but in recent years, the declining population has resulted in labor＊1 METI, “The shortage of personnel in manufacturing industries and the utilization of foreign personnel,” (July 12, 2018).
https://www.meti.go.jp/press/2018/07/20180712005/20180712005-2.pdf＊2 METI, “Electrical Safety Personnel/Technology WG,” (November 25, 2019).
https://www.meti.go.jp/shingikai/sankoshin/hoan_shohi/denryoku_anzen/hoan_jinzai/pdf/20191125_report.pdf (in Japanese)＊3 METI, “Kanto Bureau of Economy, Trade and Industry IoT and Robot Project and FY2019 Policies,” (February 2019)http://kantou.mof.go.jp/content/000226631.pdf (in Japanese)10

New Breeze Summer 2021

Copyright © 2021 Fairy Devices Inc. All rights reserved.

4. Why is Connected Worker attracting
attention for implementation of DX?

To implement DX requires building a “digital twin,”
optimizing in a digital virtual space, and then feeding it back into
execution in real space.
However, this path to implementation is not an easy one.
To achieve valuable DX with a digital twin requires a longprocess involving the following activities: (1) “Collecting” data,(2) “Curating” data into a shape that can be used, (3) “Cognition”
analyzing data and deriving inferences, (4) “Consultation” toderive measures to optimize for efficiency from the obtained
inferences, and (5) “Cropping,” to finally implement the measuresand reap the benefits of more efficient work.
To reach the final execution phase requires the worksite tobear the burden of collecting data, and costs for the enterprisecontinue. For ICT vendors like us, who are handling the effort,
maintaining the system infrastructure is a 24-hour-a-day task,
which amounts to rebuilding a society.
We have conducted earlier DX initiatives using IoT andother devices and found that there was not enough data createdby human activity to achieve efficiencies through digitalization.
Achieving such efficiencies requires DX of the whole business
and without doing so, it will not be possible to reap much value
for management. For this reason, we have now begun to focus on
using Connected Worker in various places, to digitalize the people
involved as well.

such operation requires many skilled technicians. Originally,skilled technicians would be dispatched from Japan to the location
to provide technical support, but this has become difficult due
to entry restrictions and measures to maintain safety. As such,methods to achieve DX while providing support in the developing
country from the developed country have become necessary.
The demand for digitalization measures using the“1.(1) “Remote support from an experienced technician” with
Connected Worker have increased due to COVID-19. This is theidea of enabling skilled technicians to telework.
■ Figure 8: Digitalization of devices and of people

Copyright © 2021 Fairy Devices Inc. All rights reserved.

■ Figure 9: Three workplace issues caused by COVID-19■ Figure 7: The 5 C’s of DX implementation

Copyright © 2021 Fairy Devices Inc. All rights reserved.

6. Our initiatives related to Connected Worker

Copyright © 2021 Fairy Devices Inc. All rights reserved.

5. Rapidly increasing demand for Connected
Worker due to COVID-19

In the midst of the great confusion caused by COVID-19
around the world, Connected Worker has begun to attract globalattention. A significant reason is that, due to COVID-19, it has
become more difficult to send skilled technicians for operation or
maintenance of advanced machinery that requires hands-on work.
In particular, companies in developed countries often operate
special facilities or factories in developing countries overseas, andAt Fairy Devices, where I am COO, we had requests toimplement speech recognition in a factory, which prompted us tofocus on this type of issue in on-site work, and continue testing on
work-sites.
However, most wearable devices are made based on consumer
oriented use cases, for entertainment such as games and otherindoor uses, so it has been very difficult to introduce them into
real work sites.
In particular, we often encountered issues with head-mounteddevices such as fatigue or physical strain due to the weight, or thescreen not functioning as required outdoors.
For these reasons, we decided to commit to developing ourown hardware. The “THINKLET®” device that we developed
is designed to be worn on the shoulders which is different from

New Breeze Summer 202111

prior Connected Worker devices that were in a glasses format.
We selected this design because wearing a glasses-type digital
device weighing several hundred grams on one’s head can result inheadaches and other physical strain, while even a small person cancarry weight on their shoulders easily, even up to 5 kg. Humans
feel less bodily strain when they carry objects on their pelvis or
trunk through their shoulders.
We also designed the device with special attention to handsfree operation and a simple UX, comparable to operating a TV
remote control.
As a result, we were able to realize a shape that enables
digitalization of work without significant burden on on-site
workers, either physically or from changes in work practices.
■ Figure 10: I ssues with head-mounted devices

Copyright © 2021 Fairy Devices Inc. All rights reserved.

7. “THINKLET®” features that accelerate
Connected Worker adoption“THINKLET®” has four particular features.

■ Figure 11: Introducing the Fairy Devices Ltd.
“THINKLET®”(2) 	It uses high-performance microphones, so even in a factory it
can be used with speech recognition and AI, or for creating
work records.	In fact, devices such as transceivers and pagers have beenin sites such as factories before. However, for most sites it
has not been possible to use speech recognition because thesound quality was poor, and it was difficult to hear what wassaid, even for a human. THINKLET® uses several highperformance microphones, onboard edge-AI processing andbeam forming. This enables speech to be heard clearly in theworkplace. We have also implemented speech recognition
for noisy environments, which was difficult in the past. This
speech recognition works in environments with noise of 80 dband greater, so AI can be used in factory environments.
By enabling speech recognition with noise, work records thatwere previously only done after returning to the office can becreated, and detailed evidence can be collected and delivered
while still on site.
(3) 	Use of an ultra-wide-angle camera enables the experienced
technician to provide remote support, as if they were on-site.
In the past, when the on-site worker needed help, experienced
technicians used a two-way radio or similar device to provide
support, but this had difficulties. If the on-site person said
“The green lamp is flashing,” for example, there could be morethan one green light in many cases. The THINKLET® has
an ultra-wide angle camera mounted on the front, which cancapture the on-site worker’s field of view and working area.
With this camera, the remote expert can evaluate the situation,
as if they were there, and give appropriate instructions.
T he images captured by this camera also can be used for
FairySLAM technology, which creates a 3D map from thevideo taken by the camera. This enables it to obtain location
data even inside a facility where other location informationsuch as GPS cannot be used.
(4)	Speech recognition and gesture sensors enable it to be operated
with both hands free.
■ Figure 12: THINKLET® Solution screen concept

Copyright © 2021 Fairy Devices Inc. All rights reserved.

(1) 	THINKLET® is basically an ordinary Android smartphone
inside, but it has no display and is in a form to be worn on theshoulders. Since it has no display, it is extremely lightweight
and we were able to keep power consumption very low. As
we mentioned earlier, since it is worn on the shoulders, thephysical strain on the neck is greatly reduced compared withhead-mounted and glasses-type devices.

12New Breeze Summer 2021

Copyright © 2021 Fairy Devices Inc. All rights reserved.

In contrast with office work, on-site work often requires
holding tools with both hands or wearing gloves for protection,
making it difficult to operate a keyboard or touch panel withthe finger tips. This is another reason it has been difficult topopularize digital devices for work sites.	THINKLET® has been designed to be operated while using
both hands for work, and is equipped with speech recognition
and gesture sensors. A worker can operate a drill with bothhands and ask the AI to read out from a manual, take a pictureor double check the work.
(5)	Hardware configuration is customizable for the usage andrequirements of the site.	The description above is of the basic conf iguration.
THINKLET® has also been designed to be customizable forthe desired use and requirements, with optional features suchas converting to a stereo camera by substituting the gesturesensor for a second camera, or adding lights or MR goggles
that are linked to the main unit (which we developed inhouse). This allows it to support various on-site needs.

8. Implementing “Second Brain,” for realizingworkplace DX

As mentioned earlier, in the past DX proceeded based on datafrom on-site facilities and equipment. However, factories andother worksites do not operate with just machines, and generally
there was a shortage of data regarding the non-regular work doneby people.

THINKLET® is a wearable AI that stays close to the user on
their shoulders, with accurate speech recognition and first-personview video data recording through an ultra-wide-angle camera.
Thus, it is able to collect “human Big Data” to an extent which isnot possible with fixed microphones and cameras.
It can also use LTE and other high-speed communication, sobeyond providing remote work support with video, it can be used
to implement “on-site support solutions based on analyzed data,”and “skills transfer by an AI expert technician for worker support.”
We have defined four levels of DX that can be implemented
for such solutions.
DX Level 1: BPR and data acquisition through work
digitalization
The costs incurred are not only for collecting data, the digital
devices can also be used for useful work support functions suchas providing remote support with video, and automating creationof work reports using speech recognition, by which work can bedigitalized to reduce the on-site workload.
Thus, while performing regular work as usual, work
efficiencies are increased and at the same time non-routine data
created by workers at the site will be collected automatically andnaturally.
DX Level 2: Digital work support (using data)
Using the human-created non-routine data collected in DX
Level 1, including detailed work evidence with the conditions and■ Figure 13: THINKLET® customization options

Copyright © 2021 Fairy Devices Inc. All rights reserved.

New Breeze Summer 2021

13judgments or tacit knowledge that has been revealed, processes
that were difficult previously become possible, such as skills
transfer, improved training, evidence-based status evaluation, andknowledge management.
Examples include proposing possible causes in a situation
where the pressure in a pump is lower than normal, or deriving
techniques that skilled workers do subconsciously by comparing
videos of their work with that of average workers and then using
them to update work manuals.
DX Level 3: Digital work operation reform (using AI)
To build excellent AI, it is essential to train it with anappropriate amount of fresh, precise and accurate training data,but generally it is difficult to obtain enough such data.
It has previously been difficult to gather work data from
regular day-to-day work, surrounding conditions, judgments byskilled technicians, and so on, but if we could make this possible
and use the data for training, it should be possible to build an AI
suitable for that work.
The result would enable some of the work to be automatedor optimized by the AI, such as checking completion of work or
recommending responses to inquiries from on-site and personnel
could concentrate on work that can only be done by a person.
DX Level 4: D igitalization and work integration(collaboration with AI and machinery)This is the level that most players in the AI market aspire to in
the future, and would create a world with people getting help fromAIs through digital devices, like having your own personal fairy;

like Tinkerbell in Peter Pan or Vivien in the Knights of the Round
Table.
We recommend trying not to reach Level 4 or Level 3 in oneleap, but to achieve DX in small areas with Level 1 or Level 2.

9. Creating a bright future for Japan with DXof workplace knowledge (OT)

Even with the remarkable progress in technology, it is still
people who are making the decisions in the workplace.
Japan has many excellent people with much workplaceknowledge and this is the basis for quality. I originally worked
at an American enterprise, and we often had inquiries about
exporting the advanced social infrastructure operations in Japan.
There were some cases for which we realized the export to foreigncountries, but the operational knowledge was usually confined to
the Japanese language and not digitalized, so in most cases it could
not be exported.
If an “expert technician AI” can be built using such worksite
knowledge, it would clearly be useful for increasing added value
for Japanese enterprises, facilitating efforts such as rapid setup
of overseas factories and acquiring foreign currency by creating
solutions from in-house technologies or maintenance practices.
We believe, with such knowledge confined to Japan which
is helpful for similar workplaces in foreign countries, that bypursuing DX in areas that are difficult to access by major foreign
players, we can build a future in which even Japan, with its
declining population, can make earnings in other countries. Fairy
Devices is working continuously to develop technologies andrealize workplace support AI for on-site workers■ Figure 14: A reas that Japanese venture companies can target using AI¯
Copyright © 2021 Fairy Devices Inc. All rights reserved.

14New Breeze Summer 2021