SAFETY REPORT
2020

Tier IV's Vision― Intelligent Vehicles for Everyone

We provide open access to AD technology such thateveryone, from individuals to organizations, form a sustainableecosystem that contributes to and applies AD technology toIntelligent Vehicles for the benefit of society.

2

Executive Summary
Introduction

Real-World Road Testing Activities and Approach

Tier IV aims to establish the sustainable ecosystem for autonomous driving to

In Japan, Tier IV has performed real-world road testing on public roads in about

which various organizations and individuals can contribute in terms of

50 municipalities across 18 prefectures. Outside Japan, Tier IV supports

development, and wherein society can enjoy the benefits. Tier IV offers open

customers and partners in real-world road testing of Autoware installed

access to technologies contributing to autonomous driving. and calls thisvehicles.

concept “Intelligent Vehicles for Everyone”. The greatest challenge is to ensure asocially acceptable level of safety at reasonable cost.

Real-world road testing is performed with a combination of multiple vehicle
and sensor configurations in various driving environments, also known as

Autonomous driving requires a composite platform which integrates various

Operational Design Domains (ODDs). Tier IV classifies these ODDs and provides

software, cloud infrastructure and services in addition to the in-vehicle AD

the corresponding Reference Design, which covers the combination of vehicles

systems. There is a need for a horizontally collaborative development approachand sensors for the service types of each ODD. Tier IV is showcasing its process

based on an open source platform that ensures a level of safety that satisfies

of conducting the real-world road testing, and in doing so provides methods

society at a reasonable cost. In order to realize this, Tier IV is promoting (1)for risk assessment and incident response by citing applicable cases.

Autoware*, an open source autonomous driving software, (2) AD System
Reference Designs based on ODD categorization, and (3) Deep-tech R&D of AD
systems.

*Registered trademark of the Autoware Foundation

3

Future Technology Development

Future Preparation ofEnterprise/Societal Environments

The democratization of autonomous driving technology invites a diverse set of

As outlined in Japan's national autonomous driving technology roadmap, theplayers to participate and share a knowledge base of safety in autonomous

timeline will make it is difficult for Tier IV or any other single organization todriving to a greater degree than any single organization can achieve alone. Tier IV

introduce autonomous driving technology to society while maintaining businessaims to realize this by establishing an ecosystem centered around Autoware,sustainability.

which is the open standard from Japan. Collaborating on the development ofAutoware will help industry players enhance safety to levels never realized

As a next step, government consensus is required to address challenges such as

before.

the formulation of standards for (1) government approvals and licenses(i.e.
simulation based evaluation methods for AD system driving capability), (2) AD

As Autoware is used by various players worldwide, ODDs from various parts ofsystem operation (i.e. pragmatic standards for remote monitoring personnel),

the world are collected, a common Reference Design is developed, and the edgeand (3) ensuring social security (i.e. a basic method of insurance).

cases are addressed in an effective manner. This all combines to form thetransparent open standard regarding the safety of the autonomous driving
system based on Autoware.

Based on extensive experience, Tier IV focuses its technology development
direction under the four pillars of: (1) Expansion of the perception scope and theprediction of the motion/intention of moving objects, (2) Estimation of the risk
level which is not directly captured by the sensor output, (3) Quantification of theaddressable risk levels, and (4) The Fail Safe functionality for the cases where
the risk level threshold is exceeded.

4

Contents
1.

2.

3.

Introduction

4.

Future Preparation of Enterprise/Societal EnvironmentIntelligent Vehicles for Everyone

7

Government AD Policies

37

Autoware - Architecture and Features

10

Issues related to Enterprise/Societal Environment

38

Real-World Road Testing Activities and Approach5.

Summary

6.

Appendix

40

Real-World Road Testing Activities

14

ODD Classification and Reference Design

16

Field Test Preparation and Incident Assessment

20

Drive Route Risk Assessment

23

Insights from the Risk Management Committee

25

Data

42

Cases of Test Examination and Incidents

26

References

47

Glossary49

Future Technology Development

System Safety through an Open Standard

29

Technical Issues Observed in Real-World Road Testing31Section 1

Introduction

Intelligent Vehicles for Everyone

How does AD create a better Society?

By enabling a Society...
without traffic accidents that put those involved in a difficult situation.
with a smart and efficient mobility system, utilizing cutting-edge IT.
where AD vehicles coexist with and are indispensable to citizens as an

How can we make AD a reality?

The automotive industry has evolved into a hierarchical structure where OEMs
at the top level are supported by suppliers designated as Tier 1, Tier 2, etc. AD
technology, however, is based on an integrated platform of diverse functionalitythat extends beyond the autonomous vehicle itself from the view point of
hardware, software and usage environment. Therefore, AD system development

integral part of their lives.

requires cooperative contribution by multiple players.

Tier IV, working as an inspired team, is aiming to realize “Intelligent Vehicles for

The “Intelligent Vehicle (IV)” of Tier IV expresses our aspiration for aEveryone” in society to create value for humanity from these three perspectives.

revolutionized ecosystem structure enabling access to AD technology foreveryone in the industry. In this way, individuals can enjoy the benefits of AD
technology while also contributing to its advancement.

7

To achieve social acceptance of AD, we foremost prioritize ensuring a level ofsafety that satisfies society at a reasonable cost. Tier IV will continue to

The Need for Autonomous Driving

advance cutting-edge AD technology and safety features.
The realization of AD can solve the following transportation issues.

Social Issues that can be solved with AD*

Worldwide Issues

Many victims of traffic accidents

Issues in Japan

Limitation of Public Transportation
Shortage/aging of drivers

Bad financial situation

Time wasted due to traffic congestion
Insufficient Mobility for Elderly
More elderly individuals are going out

Many disabled people cannot travel

Eldery individuals mainly use cars for mobility

Increase of fatal accidents caused by elderly drivers*Seeappendix

8Intelligent Vehicle for Everyone
Tier IV’s approach to realize autonomous driving consists of three core elements.

1.

Development and Alliance of AD Software "Autoware"

Autoware's code is open source and anyone can participate in its development.
The collaboration of Autoware and the Autoware Foundation constitutes a

involved.

TierIV's
3CoreElements

2. ODD Classification and Reference Design Provisioning

With clear ODD classifications, everyone can understand where AD fits into
complex real-world traffic environments. Furthermore, by presenting a suitable

OEMs"Reference Design" for each ODD, Tier IV simplifies the preparation process in AD
and lowers the difficulty threshold for demonstration and implementation.

3.

R&D as a Deep-tech Company

Research
Institutes

Ministries

system in which organizations and individuals around the world can get

Suppliers

ICT
Companies

Other
Partners

Deep-tech is cutting edge technology enabled by significant financial and time
investment with the goal of positively impacting society. As an academic startup
and the creator of Autoware, Tier IV aims to advance deep-tech R&D for AD
technology development in the industry.

89Autoware- Architecture and Features

In general, AD software performs perception (localization and surrounding
environment recognition), prediction, planning (path-planning and path-

Autoware's Features

Sensing

following) and final vehicle control (steering angles and brake/accelerator pedal

Collect information from external (LiDAR, cameras, GNSS,

usage). This process uses sensing information from common sensors such asetc.) and internal sensors (IMU, etc.).

LiDAR, cameras, GNSS, IMU, etc. In AD levels 1 and 2, the human driver and the AD
software work together to operate the vehicle; at level 3 and above, the AD
software basically replaces the human driver.

Autoware is aligned with these principles and Tier IV breaks down the

Localization
Localize the vehicle and sensor information within a highresolution 3D map obtained in advance.

functionality of Autoware into six parts. Details can be found on the Autoware
GitHub page*.

Perception
Detect objects, track them in time series and predict motionof moving objects.

Planning
Calculate path including the avoidance of obstacles andparking maneuvers.

Control
Generate control signals for the vehicle to follow the planned
path.

Vehicle Interface
Convert the control signals into commands according tovarious vehicle characteristics.

Map Data
Utilize high-precision 3D mapping with environmental data

*https://github.com/tier4/AutowareArchitectureProposal
10Autoware's Component Architecture11Autoware as a Platform- Open, Agile and Scalable
Autoware is free open-source software specialized for AD. It can be used byanyone for any vehicle type, with the aim of increasing the penetration of AD
into society. More than 200 companies in over 10 countries have already started
using Autoware.

Tier IV aims to realize its vision through the use of vehicles with a variety of
application/usage covering a wide range of possible ODDs. To achieve this goal,
Tier IV continues to develop Autoware with openness and expandability in mind,
and implements real-world road testing by designing systems corresponding todifferent types of environments.

Autoware as a Foundation- For Everyone
The Autoware Foundation, an international organization of 44* leading
companies in the industry, aims to promote the development of Autoware. The
development community platform, Discourse*, advances open discussions on
Autoware and drives progress on a daily basis. For example, Discourse
collaboration facilitates the knowledge accumulation of new
functions/algorithms and the promotion of compatibility with new devices. Evendevelopers without an autonomous vehicle can use Autoware and run
simulations with real data on their own computer. The provisioning of such datais enabled by Autoware's developers accumulating and sharing data from realworld road testing, and opening parts of it to the public. Through suchcollaboration Autoware contributes to the global development of AD
independent of vehicle type and region.

*As of May 2020
*https://discourse.ros.org/c/autoware/

12Section 2

Real-World Road Testing Activities
and Approach

Real-World Road
Testing Activities

Major real-world road testing activities in Japan

Tier IV promotes real-worldroad testing in Japan andacross the world to realize

Odate

Intelligent Vehicles forEveryone
Iida

Tier IV has completed real world road testing on

Sendai Airport

public roads in about 50 municipalities across 18
prefectures in Japan. It has achieved more than 70
ODD cases, and is in Japan's top tier of AD
technology leadership.

Shinjuku

Tier IV is expanding its presence worldwide,

Odaibaparticularly in Asia and North America. In the
United States, the “Intelligent Vehicles forEveryone” concept with open-source software hascome to be widely accepted and proof-of-

Fukuoka

Toyotaconcept testing with Autoware-equipped vehicles
has begun.

Sakai

Tier IV continues real-world road testing acrossthe world and is contributing to the expansion of

Osaka

Expo 2005 Aichi
Commemorative Park

safe and secure AD, backed by knowledge
collected through real life testing experience14

Major Activities Worldwide

Estonia

SwedenAt Tallin University of Technologies, 18 professors and students
created their own autonomous vehicles in a short period of time.

NEVS has launched a pilot of the PONS mobility service using Autoware enabled shuttles.

Autoware contributed to their work

8

TianjinGave a keynote speech at the world-renowned AD
conference offered by The China Automotive Technology
and Research Center (CATARC). Completed the course in an

416

AD competition.

14

8

TaiwanHong Kong

3

Hanoi

Tel Aviv
Bangkok

United States
The US Department of Transportation's Federal Highway
Administration adopted Autoware as the AD platform
for its CARMA traffic system and implemented driving
tests with Autoware based vehicles.

Supported an AV demonstration at an industrial park in Bangkok in an
ASEAN event. Implemented trial rides for senior government officials
and ministers（

No. of AWF members）

as of July 2020

15

ODD Classification andReference DesignODD 1Slow speed use cases supported by short range and minimal sensing.
No other vehicles or pedestrians.

AD dedicated envirionment

Activities

ODD classification for

Test driving

AD diffusion

AD in research institutes

ODD classification and specification is a precondition
for the AD system to work safely. Safety is ensured
because the AD technology is applied only to
specific ODDs with well understood requirements
and specifications.

Conditions defined by ODDs depend on the road

ODD 2

Use cases supported by short range sensing.
All vehicles are slow.

environment, such as vehicles, pedestrians, routecomplexity and road regulations. Tier IV classifies

Logistics roads inside factories

the ODD according to the technical complexity as
shown in the table below, which also takes player

Activitites

type and business model into consideration.

Component transportation within Yamaha's
manufacturing plant* (cart)

Tier IV implements a wide variety of real-world roadtesting within each ODD classification. We arecontinuously improving the way ODDs are classified
so that AD is introduced into society in a safe

Cyclic routes inside facilities

manner.

Activitites
A loop line bus in Moricoro Park (small-sized EV)
Facility to facility automated transportation in
Shimizu-Kensetsu's factory (small-sized EV)
A loop line bus at the Olympic Village (bus)*Implementedaseveautonomy,ajointventurewithYamahaMotor

16

Reference Design

Autoware requires connection to various

ODD 3

Use cases supported by long range sensing.
Vehicles are mid to high speed.

components, data and networks. For example, it isnecessary to establish connection to in-vehicle
hardware components, external data such as 3D

Cyclic routes in suburban areas

maps, and software offered by service providers. It
also needs to coordinate with road environment
regulations.

Activities
Public roads in Sakado-shi, Saitama (bus)

To secure all necessary connections and ensure AD

Local roads in Tokyo coastal area (passenger car)safety, every Reference Design must consider the
peripheral components and networks.

Rooted in our vision of Democratizing AD, Tier IV
creates an Autoware based Reference Design
including state of the art AD software algorithms,
and does so with close collaboration with partners

Mountain areas
Activities

in the Autoware Foundation.
One-mile transportation in mountain areas such as
Ube-shi, Yamaguchi (passenger car)
Regional transportation at Tobishima-mura, Aichi(passenger car)Tourism-oriented MaaS in Himaka-jima (bus)Urban areas
Activities
Shinjuku city area (passenger car)17

Reference Design ensures AD safenessInfrastructure
OTA Update

Regulation / Restriction
ODD Conformance

Map & Route

WiFi & 5G

Reference DesIgn

Hard Components
Sensor/Silicon

Road Regulation

Computing Unit

Inspection Scheme

By-wire/Vehicle

MaaS Management

Fleet Management

Vehicle Management

Service Providers / Users
18

Reference Design for each ODDs

ODD Optimized Reference
Designs

Tier IV seeks to achieve the optimal sensor

Sensor configuration

configuration in vehicles for each ODD

Number of

Number of

LiDARs*cameras*

1+

1+

6+

6+

6+

12+

classification. With the optimal configuration,
we can ensure safety at reasonable cost in
various real-world driving environments.

The actual design is based on the diverse set ofdetailed ODD specifications and the intended
transportation use case so that everyone can

ODD1Minimum system configuration
required for AD

easily conduct real-world road testing for AD.

Configuration for environments

ODD2

with slow and irregular moving
objects which need mid range
detection

Configuration for environments

ODD3

with uncontrollable pedestrians,
bicycles and fast-moving vehicles
which need short to long range
detection.

*See appendix
19

Field Test Preparation and Incident Assessment
Field Test Preparation
To ensure safety, Tier IV conducts extensive evaluation processes before implementing real-world road testing.

1.

Simulation

3.

Driving course risk assessment

5.

Manual/adjustment driving of test course

Validate safety by driving along the

Identify potentially hazardous events in

Manually pre-drive the course to confirm sufficient safety conditions

planned route in virtual space.

advance, and consider countermeasures thatand driver aptitude in order to finalize AD system parameters.

ensure safety.

Furthermore, pre-drive with adjusting the system parameters.

2.

Closed space test

4. Test review

6.

Perform real-world vehicle driving tests

A risk management committee reviews the

Implement the real-world driving test

on a dedicated road to check vehicle

real-world test plans and authorizes testing

depending on weather and course

functions and safety, as well as toto take place.

conditions.

Execution

conduct safety driver training.

20

Incident Assessment Procedure
Tier IV’s safety evaluation process is characterized by risk management at the occurrence of an “incident”. Although incidents in general are defined as accidents, Tier IV’s definitionof an incident is more stringent, and includes the events potentially leading to accidents. As soon as an incident is observed, Tier IV assesses the validity of countermeasures andsolutions, as well as the suspension of the real-world road testing.

Based on extensive experience, designers or implementors issue an incident report to the risk management committee when an event defined as an incident is observed.
Software/hardware engineers, safety specialists and implementers, as a team, evaluate the risk against four levels. If the highest risk level (Level 4) is observed, all internal andreal-world testing activities are suspended.

1. Evaluation of

3. Evaluation by the riskhazardous event

management committee

2. Issuance ofIncident Report

5. Design review

4. Discussions on

7. Solutions

6. Countermeasurescountermeasures
When a Level 4 incident occurs:

Suspension of all testing in and outside the company

Tier IV then discusses countermeasures to secure the necessary level of safety. Designers discuss solutions for fundamental revisions to the system implementation and/or
configuration. Tier IV removes the suspension on testing once a design review, conducted by designers and the risk management committee, confirms that safety is ensured by thecountermeasures. Ultimately, Tier IV takes steps to implement and execute the identified solutions.

Through a thorough process for incident risk management, Tier IV has established and maintained the highest level of safety during real-world road testing.

21Remote monitoring operation
The Japanese government understands the necessity for AD and is supporting its realization, but it will take some time to fully address the legal aspects due to dependencies withinternational AD frameworks. As a temporary step, ministries have published guidelines related to AD and are conducting real road driving tests under the interpretation of the currentlaw.

In parallel with the internal safety verification process, Tier IV is working together with the Ministry of Land, and the Police department on vehicle safety standards and road usage
permissions respectively as follows.

Collaboration with the MInistry of Land

Collaboration with the Police department- Regarding the road transport vehicle law- regarding the road traffic law

1. Mount the vehicle with sensors andcontrollindevices needed for AD[16]

2. Mount the vehicle with a controlling device, communications

1. Draft a test plan and consult with the police agency[17]

2. Police department cooperates with theprefectural police

device, and sensors (cameras) for remote monitoring

3. Apply to local police for road permission
3. Prior consulting with the Ministry of Land regarding
the easing of safety standards

4. Verity brake performance during remote operation insideclosed space

4. Acquire the certification to ease of safety standards
5. Verification by police officer on whether or not it is safe

5. Confirm the AD drivable route

6. Select the real world driving test course

to do testing[18]

6. Prior explanation to surrounding neighbor residents

7. Start real road testing

Since AD technology is still in the developing phase, accidents can lead to a loss of trust by society. Therefore, Tier IV cooperates with ministries in prior to conducting any testing andverifies safety and legal compliance.

22

Drive Route Risk Assessment
Collaboration with Sompo Japan
Insurance to establish driving route risk

1.

assessment methods

Study experimental conditions to ascertain potential for hazardous events

Assess test conditions

e.g. Testing period, time, objectives, driving route, driving road type (general road,
Drive route risk assessment methods are yet to be established around thefreeway, etc.), presence of test drivers/operators, presence of passengers

world. Tier IV is collaborating with Sompo Japan Insurance Inc, who has
extensive experience regarding traffic accidents. In this process, Through thisprocess, we are establishing and applying practical risk assessment
methodologies to ensure drive course safety as outlined in the following steps.

2.

Understand potential hazardous events

Potential hazardous events are assessed thoroughly by evaluating elements such as
driving conditions, test drivers/operators, persons/goods to be transported as well as
overall test operations management.

e.g. Left-turn accident involving bicycles and motorcycles, pedestrians rushingout in front of vehicle, collision with an emergency vehicle3.

Evaluate risk-level

Risks are evaluated from two angles - hazard level (damage level, impact on
business plan, etc.) and occurrence level (exposure frequency, probability ofoccurrence, risk avoidance).

Rist assessment sheet by Sompo Japan

23

4.

Establish risk reduction measures

Countermeasures are studied and designed to reduce hazardous event risks.

e.g. Introduction of escort vehicle (front/rear), driver/operator intervention whenhazardous events are observed, preparation of response manuals

5.

Prevent
Risk assessment
before driving

Reevaluate risk levels and measures against remaining risks

To reevaluate the risk of hazardous events after risk reduction measures havebeen applied, and if the risk remains at a high level, to establish additional

Monitormeasures to address.

Remote monitoring
while driving

Level IV Discovery

Accident

To make safe AD widely available, it is necessary to “monitor” driving and“prepare” for accidents, as well as “prevent” accidents through prior risk
assessment.

Tier IV, Sompo Japan Insurance and Aisan Technology have established “Level IV
Discovery”* as a solution to this problem. To achieve AD acceptance throughout
society, the solution is a risk management platform supporting AD safety andsecurity. This platform covers risk assessment, remote monitoring and insurance

Prepareas shown in the right.

Insurance forthe accident：*Level IV Discovery official website (Japanese)

https://level4-discovery.org/

24

Insights from the Risk Management Committee
Case: Setting the maximum driving speed

The initial test plan for an ODD 1 use case set the vehicles maximum runningspeed to 20 km/h. However, considering the relative speed to other vehicles andtopographic factors leading to manual/adjustment driving in the real-world roadtesting course, it was considered to increase the set speed and re-evaluate thetest examination to ensure safety.

Judgement and Response

The road-use permission standard relating to the real-world road testing states“to ensure safety and promote traffic smoothness according to road/traffic
conditions”, as well as to drive at "a speed not exceeding 20 km/h is assumed inprinciple, for the time being.“

It was judged that driving at 20 km/h was too slow due to topographic factors
and the relative speed with other vehicles, and that the maximum driving speed
should be 30km/h so that the test vehicle could safely merge with the traffic flow.
The test plan was changed and the test examination was conducted again.

24
25

Cases of Test Examination and Incidents：

Case 1

Stop order error in high-precision 3D map

An order to stop at a dangerous place was embedded on a high-precision 3D
map. However, it was found that this stop instruction was not obeyed during
manual/adjustment driving in the real-world road testing course.

Judgement and Response
The fact that the high-precision 3D map order did not work is considered aserious incident. The verification test was suspended until the cause was
identified and countermeasures were taken.

Problem and solution
Although the stop order was confirmed on the high-precision 3D map, it was
found that the stop order overlapped with other information on the user

Didn't stop

interface of the creation tool and was linked as another instruction due to thisoverlap. As a result, the stop order was not activated. As a permanent
countermeasure, the user interface of the tool has been improved so that awarning will be issued if the stop order is not activated.

26

：Case 2

Minor collision with a road-related

object during the real-world road testing
The incident occurred during the adjustment driving on the route running phaseof the real-world road testing. The autonomous vehicle lightly collided with aroad-related object when it turned left on the public road

Judgement and Response
Real-world road testing was immediately suspended, and local authorities were
notified. We then analyzed causes of the incident to discuss countermeasures forrecurrence prevention.

After having reported on the incident as well as the implemented

Left turn point

countermeasures, responsible parties gave permission to continue real-worldroad testing. The delay in the reporting process to the stakeholders brought tolight the communication issues related to incident occurrence that had to besolved.

Problem and solution

Emergency contact information was not placed in an appropriate position
leading to the delayed notification of the incident by the driver. Also, the processof contacting the relevant parties took longer than expected.

Emergency contact information was repositioned where the driver can
immediately see it and the contact procedures were changed to be more
realistic and effective with immediate effect.

27Section 3

Future Technology Development

Achieving AD System Safety through an Open Standard
Tier IV targets International standards for System Safety

Autonomous vehicles have significantly higher system complexity compared toEnsuring the safety of an AD system across many operational environments and the

traditional vehicles. The AD system of an autonomous vehicle must respond toissues observed is inefficient, let alone challenging, for an individual company.

the vehicle’s operational environment, coordinate with external service

Industry players, as a community, must gather their knowledge of AD system safety.

platforms, and enhance capabilities via software update just to name a few

Open standards are required to advance collaborative development amongst theexamples which drive system complexity. This AD system complexity leads to avarious specialized players in the AD ecosystem. Tier IV aims to achieve Systemsignificant time and financial investment making development by individual

Safety by enabling an ecosystem centered around Autoware, an AD OSS originating

industry players slow and difficult.

from Japan. Autoware enables system safety based on international standards*.

、*ISO 21448

Components of system safety

Design level

Service level

Vehicle level

TargetSystem coordination/remotemonitoring/vehicle allocation

Vehicle/surrounding
environment

System level

Software/Hardware

Component level

Software/Hardware

Risk assessment+++

Use case

Safetyanalysisanalysis

++
+

++
++
++
++

+
++

、

ISO26262、

ISO21434

UL4600, etc.

Can be covered
Main scope

Simulation analysis

+
++
++
++

Real life
testing

++
++
+
+
29

Reinforce safety and lower cost with an
Autoware centric ecosystem

ODD

Knowledge accumulation fromvarious users

Autoware

Reference
DesignAdd features
Identify risk probability
Discuss risk measures

EdgeCase

Safety verification by companies

Social implementation

1. Secure safety for features lacking in OSS
2. Add safety features lacking in ODD
3. Secure safety related to service expansion

Share as an Open standard

Social reception
30

Technical Issues Observed in Real-World Road Testing
Issue 1: Lack of ODD parameter coverage
Various real-world road tests conducted across Japan found a significantly
complex driving environment for each route. It is therefore essential to assess
real life driving environments on a case-by-case basis and to repeat cycles of
elaboration and improvement of the ODD parameters and settings. Each cycle

p32

Technology development 1:

ODD Classification and Refinement

includes, but is not limited to, the process of simulation, driving course risk
assessment and real-world road testing.

Issue 2: Inefficiency in development
due to lack of road testing classification

p34

Technology development 1:

Tier IV has performed real-world road-testing across diverse vehicle types andapplications. However, evaluation of vehicle and sensor configurations is

Expansion of Reference Designdifficult to scale as it has to be done on a case-by-case basis. Together weneed to combine and convert obtained system design knowledge from all realworld road testing into an open knowledge base. With this open knowledge
base, industry players can accelerate the development and commercializationof AD.

Issue 3: Need for enhanced Autoware functions
Real-world road testing shows that it is essential to further enhance thefeatures and functionality of Autoware. For example, continuous improvementof perception and prediction AI accuracy for localization and detection of

p34

Technology development 3:

Advancement of Autoware

objects, as well as enhanced “Fail Safe” procedures to avoid accidents in thecase of deviations from ODDs or unexpected errors.

31

Technology development 1: ODD Classification and Refinement

Tier IV has classified ODDs according to both operational business models and

AD System settings for individual ODDs include various elements such as traffic

driving environment technical requirements. These ODD classifications

conditions, road conditions, vehicles, climate, geographic characteristics, time,correspond to those defined in ”Panel on Business Strategies for Automatedetc. To improve safety, Tier IV continually breaks down system parameters andDriving”[13]. Classification of ODDs is essential to develop optimized Reference

learns from real-world testing, assessing each incident occurrence in detail. In

Designs addressing ODD requirements and form an open knowledge base fromdoing so, we continuously improve hardware and software functionality to coverreal-world road-testing in each ODD.

an increasing range of ODDs.

Traffic

Geographicalconditions

featureAmount of traffic

Uphill/downhill

Pedestrians/bikes

Curvature

RefinedWeather

Weather/temperature

ODD

Direction of Sun

VehicleHeight/width
Driving speed

Road types

Width

Time

Early morning/daytime/nightSurface
Cracks/unevenness* Panel on Business Strategies for Automated Driving: Created by the Ministry of Economy, Trade and Industry (Director-General of the Manufacturing Industries Bureau) and the Ministry of Land,
Infrastructure, Transport and Tourism (Director-General of the Road Transport Bureau) in February 2015. This “industry-academia-government” panel discusses problems and actions so that
the Japanese industry can become the leader of promoting AD and contributing to solve societal problems such as traffic accidents.

32

Correspondence with ODD classification by the government

Classification by government(restriction and environment)

Classification by Tier IV (technology and feature)

ODD1

Use cases supported by short range and minimal

Closed spacesensing. Slow speed with no other vehicles orpedestrians.

Restricted space

ODD2

Use cases supported by short range sensing. Vehicles

Driving only space

are slow speed.

Traffic environment

ODD3maintenance space

Use cases supported by long range sensing. Vehicles
are mid to high speed.

Mixed space

Classification by government

Closed spaces

Restricted spaces

Factories

Abandoned road

AIrportBus Rapid Transit (BRT)

Driving only spaces
Freeway

Traffic environment
maintenance spaces
Highway

Mixed spaces
Community roads

33

Technology development 2:
Expansion of the AD System Reference Design

Reinforcing Fail Safe functionality
It is necessary to expand Autoware functions that notify of hazards at the time

The broad adoption of Autoware is accelerating through optimized Referenceof system error occurrence. Moreover, it is required to set “Fail Safe” functions in

Designs for vehicles suitable for operation in any place and application. Tier IVadvance to address known hazardous events.

aims to expand ODD coverage across common transportation usages andfurther accumulate the experience and knowledge acquired from real-world

Tier IV can partially check whether such functions work across various ODDs

drive-testing and service operation.

through simulation. The database from the PEGASUS Project, which was run inGermany from 2016 to 2019, can be utilized to examine the functions applied toTier IV expands the scope of the AD System Reference Design based on thegeneral use cases. The SAKURA Project, a similar project which began in Japan

experience accumulated across ODDs. The AD System Reference Design isduring 2018, can be utilized to address use cases applicable to the road

specified and optimized against combinations of ODDs and vehicle applications.

environment in Japan.

In addition, Tier IV seeks opportunities to openly share knowledge with otherindustry players.

In addition, in order to comply with situations such as ODD deviation and otherunexpected hazardous events, low probability hazard events must be translated
into use cases, including those events that cannot be realized via simulation.

Technology Development 3:
Advancement of Autoware
Improving the accuracy of "perception/prediction" and"planning" functionality“Perception/prediction” and “planning” are playing an important role in thequality of the overall AD functionality. In this context, accuracy improvement
requires advanced AI research, and as there is a shortage of AI researchers in
the industry, collaboration with academia is indispensable. AD systemdevelopment requires collaboration amongst specialists in areas such AI,
software engineering and automotive engineering. Tier IV improves theaccuracy of “perception/prediction” and “planning” based on a multidisciplinary
effort spanning business and academia.

：：*PEGASUS ProjectA project aimed to define the safety evaluation methods suitable for German OEMs. The project is based on a public-private partnership and established by 17 organizations including BMW, Daimler, Audi, and Bosch.

https://www.pegasusprojekt.de/en/about-PEGASUS*SAKURA Project

Project aimed to develop the safety evaluation methods for AD suitable for Japan. It is based on a public-private partnership and promoted in collaboration with overseas projects including the PEGASUS project.

34

Expanding Reference Design and ODDs classification using theknowledge from real-world road testing

less

ODD 1

Transportation of people

moreTransportation ofgoods

AD dedicatedWide range of vehicles can comply because initial testing is done

environment

in a closed access road before real-world road testing

Micro mobility

Cart

Microbus

AMR

Cart

Logistics roads

ODD 2

inside factories
BRT/
Cyclic routes insidefacilities

Passenger car
Truck(last mile)

Cyclic routes insuburban areas

ODD 3

/

Cart (last mile)

Mid-sized bus

Mountain areas/
Urban areas

Wagon/van(last mile for deliveries)

DomainTierIVhave
covered

DomainsTierIVhasn't
coveredyet

35Section 3

Future Preparation ofEnterprise/Societal Environment

Government Policies for AD
In May of 2020, the Japanese government’s Panel on Business Strategies in

The next step is to discuss methods for ensuring safety in AD from the following

Automated Driving published the roadmap for AD based on various roadperspectives; (1) design of objective standards for driving environment

environments and service types[13]. To realize the AD roadmap, the government

conditions and (2) discussion on safety standards for AD vehicles. Tier IV follows

first classifies driving environments.

the government guidelines and ensures safety by challenging and addressing theissues of technology and the enterprise/societal environment.

The Japanese government is ensuring safety in realizing the AD roadmap

Safety levelytefaS

Fully autonomous

AD technology

Safety secured by driving

Safety secured by vehicle

Safety secured by ODD

Safety secured by human driving

environment

37

Issues of Enterprise/Societal
Environments
Unlike overseas, real-world driving tests withmultiple vehicles are not conducted in Japan
In Japan, real-world road testing is performed in many diverse areas, and theapplication for testing must be submitted to ministries and local governmentsin every instance of testing to be conducted. Testing is therefore conductedwithin a short period of time using only a small number of vehicles.

On the contrary, in the US more than 80 companies have conducted driving
tests on public roads. As of June 2019, these companies have used more than
1,400 vehicles in 37 states over an extensive period of testing[15]. While thiscontributes to the accumulation of aggregate driving data in the US, the testingareas and vehicle types for real-world road testing are limited per company
and learnings are not shared openly.

38

It is essential to establish a common environment for AD System development andoperation across the boundaries of individual companies

Problem 1: Lack of simulation standards for authorization

Establishment of authorization standards

Human drivers are awarded licenses based on their driving capability. These

The driving capabilities of AD Systems are reproducibly quantified under specificcapabilities are tested under specific environments with clear standards ofscenarios by simulations. It is necessary to develop evaluation methods in simulation

accepted performance. However, for AD Systems, there is as yet nofor the various aspects of driving capability and the respective authorization standards.

authorization standard for driving capability based on simulation scenarios withThis will enable a shorter time to authorization, allowing for a greater number ofagreed upon performance criteria. Driving authorization is awarded tovehicles to be used in real-world road testing, and therefore, speeding thecompanies on a case-by-case basis.

development of sophisticated AD technology.

Problem 2: Lack of standards for operation

Establishment of rational operational standards

An operational system does not have standards for methods of dealing with

The system of “Fail Safe” functions, human drivers and remote monitoring by humans

hazardous events. Companies individually consider methods to deal with suchmust deal with a vehicle’s deviation from ODD specifications or unexpected errors. It

events from the perspective of the probability of such hazardous eventsis important to set rational standards for the personnel systems and operational

occurring, as well as the types of personnel systems and operational methods

methods established to respond to such situations. This will enable AD systems tofor remote monitoring.

curtail workforce shortages and solve societal issues such as the lack of drivers.

Problem 3: Lack of insurance methods in case of accidents

Establishment of basic insurance

Under any authorization standard or operational setting, unexpected risks exist
for both human drivers and AD systems. For human drivers, automobile liability
insurance exists as a basic insurance offering. However, basic insurance has yet
to be established for AD systems.

The same as for human drivers, an AD system requires basic insurance. The insurance
must cover the inevitable risk that the AD system operates outside the driving capability
standards for which it was authorized. These risks remain even with the preparation of“Fail Safe” functions and remote monitoring systems.

39Conclusion
Autonomous vehicles for everyone ――

Founded 5 years ago ― Tier IV has continuously aimed to realize the goal ofIntelligent Vehicles for Everyone. To achieve this goal, we develop AD technology
with our open source AD software, “Autoware”. We have performed many realworld road tests spanning various ODDs across the world. Our experience in realworld road testing is the best-in-class in Japan.

This Safety Report provides an overview of ODD classification and AD System
Reference Design development against ODDs. Based on this methodology, Tier IVshares with industry players real-world road-testing processes to ensure AD
System safety. This report also addresses common issues in the areas of AD
technology development and the business/societal environment for acceptanceof AD technology and services.

AD systems extend well beyond the basic autonomous vehicle. AD Systems arebuilt upon a unified platform of hardware and software applied to usage
environments. It is essential that industry players contribute to AD’s collaborative
development across horizontal technology domains. This collaboration will leadto a sustainable ecosystem and enable the prompt realization of AD in society by
establishing socially acceptable safety at reasonable costs.

We believe in AD’s potential to decrease traffic accidents, provide increased
mobility for the disabled and elderly, and support the sustainable use of publicresources for transportation.

Tier IV continues to advance “Intelligent Vehicles for Everyone”.

40

APPENDIX

DataSocial Issues that lead to the need of AD

WorldwideJapan

Traffic fatalities (people/year)

1.4 million

4000

Traffic injuries (people/year)

20-50 million

529,000

Time wasted in traffic jams

20% of driving time

40% of driving time

36 million

140,000

Number of blind people

Heavy vehicle class 2 licensees-20% ('09-'19)

Number of taxi drivers-27% ('08-'18)

Average bus driver's age

51.2 (+8.7 from mean)

Average taxi driver's age

59.9 (+17.4 from mean)*97% of fatal accidents are caused by human errors*effective job offer rate in driving industry is twice as that of the average*grew by +4.4 in the last 10 years

42Number of deficit operator
Abandoned routes (km)Bus

Regional railway

69%

76%

13,391 ('07-'16)895 ('00-'19)

Local cities

Percentage of elderlies who mainly use cars

Metropolitan areas

36%

55%

Percentage of elderlies going out+21% ('87-'15)

Percentage of fatal accidents caused by elderlies+16% ('08-'18)

43

Reference Design for each ODD

LiDAR configuration example

Camera configuration example

ODD 1

LiDAR (localization)

1 LiDAR
Localization/360° long range object detection

Camera (remote monitoring)

1 camera
Remote monitoring of front side44

LiDAR configuration example

Camera configuration example

ODD 2

Camera (remote monitoring)

LiDAR (localization)

LiDAR (short range)

Camera (object detection)

LiDAR (mid range)

Camera (signal detection)

6 cameras
6 LiDARs

360° remote monitoring (4)Localization/360° long range object detection (1)Object detection of front side (1)360° close to mid range object detection (5)Signal detection of front side (1)45LiDAR configuration example

Camera configuration example

ODD 3

LiDAR (localization)

LiDAR (mid range)

LiDAR (long range)

Camera (remote monitoring)

6 LiDARs

12 cameras

Localization/360° super long range object detection (1)360° remote monitoring (4)Mid range object detection of rear side (1)360° object detection of front side (6)Mid to long range detection of surrounding vehicles andSignal detection of front side (2)Camera (object detection)

Camera (signal detection)

humans (4)46Reference[1] "Road traffic injuries", World Health Organization, 2020.
https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries ( 2020-5-14)[2]『令和元年版交通安全⽩書』 内閣府,, 2019.

https://www8.cao.go.jp/koutu/taisaku/r01kou_haku/index_zenbun_pdf.html (2020-5-14)[3]『第 回 ⾃動運転等先進技術に係る制度整備⼩委員会 配付資料 』 国⼟交通省
13,, 2018.

https://www.mlit.go.jp/policy/shingikai/jidosha07_sg_000052.html (2020-5-14)[4]『⾼速道路を中⼼とした「道路を賢く使う取組」の基本⽅針』 国⼟交通省,, 2015.

https://www.mlit.go.jp/policy/shingikai/road01_sg_000218.html (2020-5-14)

[5]

『

2050

年に世界で 億1

万⼈が失明』

1,500, Medical Tribune, 2017.

https://medical-tribune.co.jp/news/2017/0818510031/ (2020-5-14)[6]『運転免許統計（令和元年版及び平成 年版）』 警察庁
21,, 2020/2010.

https://www.npa.go.jp/publications/statistics/koutsuu/menkyo.html (2020-5-14)[7] ”TAXI TODAY in Japan 2020”,

⼀般社団法⼈全国ハイヤー・タクシー連合会, 2020.

http://www.taxi-japan.or.jp/pdf/Taxi_Today_2020.pdf (2020-5-14)[8]『令和元年度 第 回 過疎問題懇談会 資料 』 総務省
36,, 2019.

https://www.soumu.go.jp/main_sosiki/jichi_gyousei/c-gyousei/2001/kaso/02gyosei10_04000065.html (2020-5-14)

47

[9]

『若者敬遠、⽼いるバス運転⼿ ⼈材不⾜で路線縮⼩も』 産経新聞,

, 2019.

https://www.sankei.com/life/news/190723/lif1907230015-n1.html (2020-5-14)[10]『未来投資会議（第 回）配布資料 』 ⾸相官邸
32

1

,

, 2019.

http://www.kantei.go.jp/jp/singi/keizaisaisei/miraitoshikaigi/dai32/index.html (2020-5-14)[11]『第 回 地域交通フォローアップ・イノベーション検討会 配付資料 』 国⼟交通省
15,, 2018.

https://www.mlit.go.jp/sogoseisaku/transport/sosei_transport_tk_000093.html (2020-5-14)[12]『⾃動運転の公道実証実験に係る道路使⽤許可基準』 警察庁,, 2019.

https://www.npa.go.jp/bureau/traffic/selfdriving/20190905jidouuntenkyokakijyunkaiteiban.pdf (2020-5-14)[13]『⾃動⾛⾏ビジネス検討会 報告書「⾃動⾛⾏の実現に向けた取組報告と⽅針」

Version 4.0』 経済産業省,, 2020.

https://www.meti.go.jp/shingikai/mono_info_service/jido_soko/20200512_report.html (2020-5-14)[14]『官⺠ 構想・ロードマップ
ITS』 ⾼度情報通信ネットワーク社会推進戦略本部・ 官⺠データ活⽤推進戦略会議

2019

,

, 2019.

http://www.kantei.go.jp/jp/singi/it2/kettei/pdf/20190607/siryou9.pdf (2020-5-14)[15] "Over 1,400 self-driving vehicles are now in testing by 80+ companies across the US", TechCrunch, 2019.
https://techcrunch.com/2019/06/11/over-1400-self-driving-vehicles-are-now-in-testing-by-80-companies-across-the-u-s/ (2020-5-14)[16]『⾃動運転⾞の安全技術ガイドライン』 国⼟交通省,, 2018.

http://www.mlit.go.jp/common/001253665.pdf (2020-5-14)[17]『⾃動運転の公道実証実験に係る道路使⽤許可基準』 警察庁,, 2019.

https://www.npa.go.jp/bureau/traffic/selfdriving/20190905jidouuntenkyokakijyunkaiteiban.pdf (2020-5-14)[18]『遠隔型⾃動運転システムの公道実証実験に係る道路使⽤許可の申請に対する取扱いの基準 』 警察庁),, 2017.

https://www.canva.com/design/DAEDVJPifnU/wkGsKMpZGgU25VbeNkB4aw/edit (2020-5-14)

48

Glossary
ODD

Operational Design Domain: the preconditions (e.g. driving environment) for autonomous driving systems to work safely.

Reference Design

The blueprint that Tier IV provides, along with other solution providers, to support the implementation of Autoware.

Deep Tech

The cutting-edge technology developed through great financial and time investment, which has a beneficial impact on society.

Edge case

The rare, hard-to-predict outlier case which can lead to the hazardous events.

Fail safe

One of the concepts which should be taken into consideration in the design of devices and systems to build a mechanism
which transfers an autonomous vehicle to a safer condition when malfunction/abnormality occurs.

LiDAR

Light Detection and Ranging. A sensor that measures the distances to the targets using a beam of light, which is essential for AD.

GNSS

Global Navigation Satellite System. An umbrella term for the positioning systems that use satellites (e.g., GPS).

IMU

Inertial Measurement Unit. A device that measures vehicle's angular velocity and acceleration by combining a 3-axis gyroscope andaccelerometer.

AMR

Autonomous Mobile Robot. A robot for internal transportation that avoids people or other obstacles autonomously.

BRT

Bus Rapid Transit.

49

Co-editor

EditorTier IV, Inc.
Jacom Bldg., 1-12-10
Kitashinagawa, Shinagawa-ku,
Tokyo, 140-0001, Japan

Distribution, quotations and duplications – even in the form of extracts – for third parties is only permitted upon our prior written consent. The text and charts used in this report do not thoroughly represent the complete content of the report.

50