Real-time Detection of the “Three Cs” with Image Analysis“COVID-19 AI Image Analysis Solution”
Takashi Yanagihara

Representative Director, CEORidge-i Inc.

1. Introduction

Efforts to stop the spread of COVID-19 and restore stableeconomic and social life continue. Negative impacts on the entire
market are huge, and annualized GDP is expected to drop 20%
or more. However, there are also industries that have prospered
during the pandemic, and solutions needed for the “New Normal”
are f lourishing, with demand for online meeting and SaaS
business tools for remote communication and remote work, robotic
technologies for remote operation and automation, and services
such as online shopping and delivery for at-home consumers.
With a mission to “Solve social and customer issues and create
new business and a new society using advanced technology,”
Ridge-i Co. Inc. has provided unique AI solutions in wideranging fields and has had much success* with AI image analysis,
focusing particularly on deep learning.
The majority of our members are either practitioners andresearchers in machine learning, or from strategic consulting
firms, pursuing ideas from a broad perspective (hence “Ridge”)
through knowledge synergy among specialists from various fields.
＊ One example of success in development is a technology toautomatically colorize black-and-white video, developed incollaboration with NHK Art Inc. In collaboration with Ebara
Environmental Plant Co. Ltd., we also developed a deeplearning system to recognize garbage features. It has beenin use for over one year as part of an “Automatic garbage
identifying AI crane” at a waste incinerator facility operated
by Funabashi City. We also have several initiatives using
AI to analyze satellite data, including a land-slide analysis
AI developed under contract with JAXA, and SAR image
analysis for the major oil spill that occurred near Mauritius,
as reported on Yahoo! News and other major media. We havebeen awarded the Fourth Space Development and Utilization
Grand Prize by the Minister of Economy, Trade and Industry(METI) for our utilization of advanced technology to solve
societal issues.
In this article, we discuss our “COVID-19 AI Image Analysis
Solution,” which is able to detect the three C’s: “Closed spaces,”“Crowded places” and “Close-contact settings”; from camera
images in real time. We developed and released it in only twoweeks after the state of emergency was declared in April 2020.
We discuss its features and future prospects, as well as future
requirements for AI technology and organizations.

2. Background to “COVID-19 AI Image
Analysis Solution”

On April 7, 2020, when many countries were just beginningto implement measures such as social distancing to prevent thespread of novel coronavirus infection, seven prefectures, including
Tokyo, declared a state of emergency. On April 16, our Chief
Research Officer (CRO), Yoshitaka Ushiku and I met to discuss
for the first time whether there was anything we could do. That
was when the concept for “COVID-19 AI Image Analysis
Solution” took shape. The next day, we met with our researchengineers and decided on an architecture, and our engineers
volunteered to take on parts to be worked on in their spare time.
Development was completed on May 1st, and the whole solution
was released on that day. All work from conception to release wasdone in approximately two weeks.

3. Measuring closeness, crowdedness andcrowd size from camera video

The “Video Analysis AI Solution for COVID-19,” releasedin May 2020, is an AI solution that accurately detects the number
and location of people in network cameras in almost real-time,and analyzes the distance between people, the degree of crowding,and the flow of people in crowds. It consists of the following threeelements.
In addition, we offered a course on AI fundamentals (4) tohelp participants make effective use of their home time during themajor holidays under the emergency declaration and to deepen
their knowledge of AI and machine learning.
(1) C rowd counting, density estimation, and time-series
measurement of passers-by (for street cameras, commercial
facilities, and tourist facilities)(2) Close-contact alerts (for offices, hospitals, shops, etc.)(3) Person tracking and Re-ID(4) AI fundamentals lecture, free-of-charge on YouTube(1) is a system that analyze video from cameras in the street
or commercial areas, displaying congestion conditions as a heat
map and counting passers-by and the size of groups in the video
in near-real-time. It can also show a graph of the results, whichshows the number of passers-by over time. Currently, a demo of
the crowd-size counting module is available on YouTube, used
to analyze real-time video from areas of Hamamatsu City inShizuoka Prefecture (see link at the end of the article). This has

New Breeze Summer 2021

5

been operating continuously since May, using AI functions thatanalyze images from cameras installed at fixed locations in thecity. It locates passers-by and areas with congestion in the video,
displaying data on the web page as a heat map or as a graph ofdaily and hourly traffic.
The camera stream is processed by an AI installed on thecloud, and the analysis is achieved with almost no delay from thelive video.
Currently, the system is accurate enough for practicalapplications in large facilities such as stadiums, commercial
facilities, or public offices with many people coming and going.
In the future, we intend to also support night-time operation, andwe are also already preparing to provide versions that run well on
Android and edge devices.
(2) assesses the separation of people in more closed
environments such as offices, hospitals, or shops; in order todetect and alert the user when people get too close, or whencongestion occurs as the density of people exceeds a certain level.
For example, in an office with a large number of staff movingaround, if the distance between them becomes less than 1.5 meters(the distance can be set freely), an alert is issued in real time.
It is also possible to notify by sound or vibration. In particular,

the distance between people is measured with high accuracy by
capturing the position of each foot using AI for posture estimationand person detection.
(3) performs person tracking and re-identification, identifying
each person by analyzing their face, bone structure, clothing,walking style, etc. without a need for prior registration of users.
The mechanism assigns a unique ID to each person when theyfirst appear on the camera image. Once this ID has been assigned,
it will be applied to that person even if they leave the image andreappear later. As such, the technology can be used to authenticatethat it is the same person (Figure 1). This person tracking function
could be used to check a person’s past movements from camera
video if, for example, they are diagnosed with COVID-19, toidentify who they were most in contact with or locations requiring
careful disinfection. This could also prove itself useful in order toperform procedures more efficiently after an infection is detected.
By using the kind of solutions that we provide, it is possible todetect and avoid “dense” situations in various environments in realtime. In particular, since they are based on camera image analysis
technology, they have advantages over mobile base station data(Mobile Spatial Statistics), which operates on a coarse mesh, andcan cover indoor environments where GPS is weak. They can also

■ Figure 1: Person tracking and Re-Identification AI. Assigns IDs without prior registration. Assigns ID by making use offeatures such as posture, body type, etc. so that the same ID is assigned even after the subject puts on a jacket.

6New Breeze Summer 2021

determine whether masks are being worn through image analysis.
We have already had inquiries from public offices, retail chains
and automobile dealerships and we are conducting trials withsome of them.
Our fourth initiative (4) is unique as a solution. On theassumption that the number of on-line students will increase
because they are spending more time at home, we have beenoffering basic AI training lectures free-of-charge on our YouTube
channel since May. The lectures cover a curriculum conceived
by our CRO, Yoshitaka Ushiku, to provide an overall grasp ofAI. They include ten basic-lecture videos following the historyof machine-learning development, from the birth of AI to thepresent. This course is also used as a tool to build a correctcommon understanding of AI with our customers.

Our AI development for posture estimation and person
identification is at the top level in the world in academic
benchmarks, and in particular, our proprietary deep learning
method for posture estimation has achieved SOTA (State of theArt) in public benchmarks (as of June 2020).
A single deep learning is rarely enough to solve a real-worldproblem, but by combining multiple deep learning models, users
can experience an end-to-end AI solution.

given limited resources and unexpected troubles, it has been
more difficult to work concretely based on this mission than weimagined. There are other companies with similar missions, but
it seems that most of them were not able to maneuver under thedeclaration of emergency and its effects, or to adjust for remote
work and other measures. We also had planned projects just before
the declaration of emergency, which changed drastically becauseof it, and we were forced to respond quickly. However, taking
action under such conditions is the hallmark and raison d’etre of aventure company, and we managed to overcome these problems bya good margin.
In addition, in order to respond quickly to unpredictablesituations, it is important to create robust elemental modules andtechnologies such as the ones mentioned earlier, so that we cancombine them flexibly to provide an AI that can answer specificproblems. Beyond just advanced development skills, the ability
to understand real-world problems, divide them in elemental
components and translate them into technical requirements is also
crucial.
Even if we see what needs to be done technically to solve
the problem, it can be difficult to reach the decisions necessaryto execute it. In this case, our CRO, Yoshitaka Ushiku, and I
went ahead based on our own judgment, but we could not havesucceeded without our volunteers’ support. We strongly felt theimportance of our mission and our organizational culture. It
was important to decide roles quickly and pursue multiple tasks
simultaneously without waste of time, not only by top-down
decisions but also by parallel and organic decisions taken at alllevels of hierarchy.
The four solutions we provided this time are not the ultimate
solutions to problems, but they have provided an excellent
opportunity to challenge the technologies and the organizational
culture we have cultivated over four years. I think the greatest
assets we have gained are a greatly increased awareness of ourdevelopment processes, of what sort of AI technical elements
can be modularized to provide functionality quickly and withflexibility, and the real experience of creating a solution within aperiod of only two weeks.

5. New social challenges from the COVID-19
pandemic

6. AI technology requirements and company
qualities required in the future

4. AI technologies used in the solution

Our COVID-19 AI Image Analysis solution’s strengths are its
exceptional accuracy and inference speed in each of the technicalelements, as well as how they are integrated. The main technicalelements used in the solution are described below. While each ofthem alone could be considered AI, we refer to combinations ofmultiple elements as a single solution.
・Pose estimation Deep Learning
・Human Detection Deep Learning
・Human attribute classifier Deep Learning
・Human and Object Tracking Technology
・High-speed, stable video stream processing
・Crowd Counting Deep Learning

One thing we felt keenly in creating this “COVID-19 AI
Image Analysis Solution” was the importance of preparing systems
such that technologies can be provided immediately when needed.
To solve immediate needs, quality is of course necessary, but speed
is required above all else.
As mentioned in the introduction, our mission is to “Solvesocial and customer issues using advanced technology andcreate new business and a new society,” but practically speaking,After the announcement of the solution, we received not
only inquiries from commercial facilities, but also requests forinterviews from various media groups, including NHK. It wasparticularly impressive that we were asked not only about thedetails of the functions and technology, but also about why wedecided to create such a solution.
Six years have already passed since the tertiary AI market in
Japan began gaining notice in 2014. With a large amount of AI

New Breeze Summer 2021

7

ventures in the market, it is difficult for a company to shine basedon its AI technology alone. The market also has a reputationthat many projects do not go beyond the PoC stage, resulting
in mixed feelings of hope and disappointment. According tomaterial published by the Ministry of Economy, Trade andIndustry (METI) in March 2020, 97% of AI related projects areabandoned before completion, which is quite surprising. We havesummarized the obstacles faced by some representative projects
below (Figure 2).
Considering the level of maturity of this market, companies
that have been developing AI thus far will need to make drasticchanges. They are already expected to provide not only promising
AI prototypes, but also successful integrations that can be used inpractice.
Social and business needs are also changing prompted by
the COVID-19 pandemic, but the New Normal has not settled
yet. Under such conditions, we must assume that there are still
no precedents for some user needs or how to address them, andventures must find agile solutions quickly, by experimenting
quickly and not being afraid of failure.
In the New Normal, the need will likely increase for solutions
characterized by keywords such as remote monitoring, remote
operation, semi-automation, and location-free. AI technology will

be a necessary element of such solutions, but many problems inrobotics, communications, edge inference, sensing and other fields
cannot be solved with AI alone, making it increasingly importantto have partnerships with enterprises in various other fields.
The qualities that a company must have in such anenvironment include (1) ability to sense and adapt flexibly to newneeds in society, (2) rapid development, deployment capabilities
and technical assets to respond quickly to urgent issues, (3) design
and technical capabilities to go beyond planning to successfully
deploy their solution and produce results, and (4) ability tocollaborate with partners in solving problems rather than only
acting alone.
At Ridge-i, we plan to maximize the AI technology assets andpersonnel that we have cultivated to provide solutions that produce
major benefits for our customers, replacing the negative impacts
brought on by COVID-19. We encourage partner companies andindividuals to contact us, to propose and implement solutions for
the New Normal together.
Related Links
Ridge-i Inc. https://ridge-i.com/
Hamamatsu City, Kajicho-dori, Live demo of counting crowd numbers(Ridge-i YouTube Channel) https://www.youtube.com/channel/UCTBaJrfibBNosxCVEjPnYWw■ Figure 2: Bottlenecks to introduction of AI. 97% of AI projects are abandoned uncompleted.

8

New Breeze Summer 2021