Ellipse 3 | Performance Assessment Report
In automotive application

Product on test
Test Environment

AUTOMOTIVEELLIPSE SERIES
Ellipse-D v3

PREAMBLE
For the purpose of this report, SBG
Systems conducted a series of roadtest campaigns to assess the performance of the Ellipse-D under various
environmental conditions typical of
automotive applications.
The Ellipse-D is a cost-effective
entry level INS in the portfolio of
SBG Systems well suited for automotive usage.
TESTED PRODUCTS

Ellipse-D, miniature inertial
navigation system (INS) for low
dynamics and robust headingwith a full-GNSS dual-antenna
RTK receiver.
TESTED PERFORMANCE

Roll, Pitch, Heading, andPosition
DATE, PLACE

June, 2021 in Carrières-surSeine, France

We assessed the performance of
the Ellipse-D in real time receiving
RTK corrections and odometer input
and in standalone mode (not aided).
Measurements were also processed
oﬄine using SBG Systems Qinertia
PPK software.
The reference system is the SBG
Systems Horizon FOG IMU with Navsight-T receiving RTK corrections
and odometer input. For this FOG
unit, the post-processed accuracy
obtained is better than 0.004° on
attitude and centimeter-level on
position. An RTK base station was
installed in optimal conditions witha clear view of sky and logged data
during the entire test campaigns.

MOUNTING
Both Ellipse-D (RTK, Standalone) and the Horizon IMU
were rigidly mounted on aplate in the car. GNSS leverarms, the vehicle center ofrotation as well as pitch/
yaw alignments within thevehicle frame were accurately
estimated using Qinertia PPK
software with a Tightly Coupled computation.

TEST CONDITIONSDEVICES UNDER TEST (DUT)

SensorPosition (2d) Accuracy

Attitude Accuracy

Heading Accuracy

Ellipse-D production units were usedin different operating modes:

EllipseRTK

0.01m(0.8m @ 10s)
(5.5m @ 60s)

0.05°
(0.05° @ 10s)
(0.10° @ 60s)

0.2°
(0.2° @ 10s)
(0.3° @ 60s)

EllipseSinglePoint

1.2m(1.7m @ 10s)
(7.0m @ 60s)

0.1°
(0.1° @ 10s)(0.15° @ 60s)

0.2°
(0.2° @ 10s)
(0.3° @ 60s)

EllipsePPK

0.01m(0.1m @ 10s)
(1.0m @ 60s)

0.02°
(0.08° @ 10s)
(0.1° @ 60s)

0.1°
(0.1° @ 10s)
(0.2° @ 60s)

SensorPosition (2d) Accuracy

Attitude Accuracy

Heading Accuracy

The reference system is the Horizon
IMU with Navsight-T receiving RTK
corrections and odometer input.

Horizon0.01m(0.01m @ 10s)
(0.05m @ 60s)

0.004°(0.004° @ 10s)
(0.005° @ 60s)

0.008°(0.008° @ 10s)
(0.010° @ 60s)

Specifications given for Tight-Coupling PPK with Automotive motion
profile @1σ.

The specifications of the Horizon
FOG (Fiber Optic Gyroscopes) IMU
make it a perfect candidate to obtain» Ellipse-D v3 receiving RTK
corrections and odometer input(RTK+Odo)» Ellipse-D v3 not receiving RTK
corrections, nor odometer input(Single Point)» Ellipse-D v3 tightly coupled postprocessing with oﬄine GNSS data
(PPK)
Specifications given @1σ.

REFERENCE

a tightly coupled PPK solution and to
be used as a reference for our tests.

CONFIGURATION & ACQUISITION SETUP
Each sensor is rigidly mounted on a
dedicated fixture attached to thetest vehicle. The lever arms were firstmeasured and then precisely re-esti-

Ntrip connection
Modem Teltonika

Serialdata
loggermated with Qinertia post-processing
software (see detailed explanations
further in chapter “Lever arms estimation with Qinertia”). The diagramin Figure 1 depicts the connections to
the external aiding inputs, the datalogger and the splitters needed toshare the dual-antenna system.

Tallysman Verostar
GNSS Antennas

Tallysman Verostar
GNSS Antennas

Slitter x4 GPS
Networking (A1)

Slitter x4 GPS
Networking (A2)

Ellipse-D
V3

Ellipse-D
V3

Figure 1 - Systems connection

Navsight
HorizonOdometerPegasemRECORD DATA WITH SBGDATALOGGER
SbgDatalogger is a companion
software tool supplied by SBG Systems to record binary data from thetwo units under test and to feed
RTCM corrections from a NTRIP
server to the device RTK + Odometer.
It allows to use a single serial port tosend RTCM corrections while retrieving and logging measurements froman Ellipse.
The Figure 2 shows the main interface.
The left panel controls and monitors
the connection to the NTRIP server.
The right section displays various
statuses information separately foreach connected Ellipse.
The first Ellipse is receiving RTCM

corrections with an RTK fixed GNSS
solution whereas the second one isonly providing a standalone GNSS
position.

BASE STATION SELECTION FOR
PPK
The base station used for PPK
computations was SBGS, from theprovider IGNRGP (Figure 3). The
maximum baseline for all the tests
was 13 kilometers.

LEVER ARMS ESTIMATION WITHQINERTIA
Qinertia is very practical to estimate
GNSS and odometer lever arms. For
automotive applications, Qinertia

can also estimate the vehicle centerof rotation as well as pitch/yaw alignments within the vehicle frame.
We performed an acquisition whiledriving “8-shape” patterns underan open sky environment (Figure 4).
For optimal lever arm estimation, it
is always advised to expose the INS
to dynamic motions in good GNSS
conditions.
The acquired data has been postprocessed in Qinertia using a Tightly
Coupled PPK computation to estimate accurately the INS installation
lever arms and alignment.
Once the computation is completed,
Qinertia automatically reports therefined mechanical installation parameters with QC indicators.

Figure 2 - sbgDataLogger GUI

Figure 3 - Maximum baseline value for all acquisitions

Figure 4 - «8» shape pattern acquisition

OPEN SKY ACQUISITIONDESCRIPTIONThis acquisition is mostly performed
in Open Sky environment between
13:38 to 14:34 UTC.

DUT PERFORMANCE
Error analysis compared to reference
An analysis is carried out on all thedata to obtain the quality indicators.
The table below shows the percentiles
at 68 %, 95 % and 99.7 % on errors
between each DUT and the reference
measurements.

2D position 68%
0.013

2D position 95%

2D posiAltitude
tion 99.7% 68%

Altitude95%

Altitude
99.7%

0.033

0.221

0.057

0.072

0.136

0.639

1.069

2.212

0.904

1.516

2.111

0.006

0.017

0.040

0.007

0.017

0.052

ELLIPSE-D PPK 0.006

0.017

0.038

0.007

0.016

0.047

Yaw 68%

Yaw 95%

Yaw 99.7%

0.087°

0.217°

0.481°

0.094°

0.208°

0.362°

0.072°

0.169°

0.428°

0.055°

0.139°

0.280°

Percentile Error
ELLIPSE-D
RTK+ODOELLIPSE-D
Standalone
ELLIPSE-D
PPK+ODO

Roll /
Roll /
Roll /
Percentile Error Pitch 68% Pitch 95% Pitch
99.7%
ELLIPSE-D
0.015°
0.036°
0.077°
RTK+ODOELLIPSE-D
0.030°
0.089°
0.179°
Standalone
ELLIPSE-D
0.006°
0.013°
0.029°PPK+ODO0.014°
0.026°ELLIPSE-D PPK 0.006°

Figure 5 - Cumulative distribution error (Open Sky)CUMULATIVE ERRORS FIGURESThe graphs above illustrate cumulative distribution errors for 2D Position, Heading, Pitch and Roll i.e. theerror percentage below a value. Herefor example, the first graph showsthat :» 95 % of Ellipse RTK + Odometer
Real Time position errors are
below 3.3 cm.
» 95 % of Ellipse Single Point RealTime position errors are below 107
cm.
From these results we retain that theaccuracy of the solution obtained in
the field are totally in line with SBG
Systems specifications (see chapter“Devices Under Test”), more precisely:» Roll, pitch and heading accuracies
are within specifications forReal Time Single point solution,
Real Time RTK solution and PPK
solution.
» Position accuracy is also perfectlyin line with specifications forReal Time Single point solution,
for PPK and for Real Time RTK
solution.
The Ellipse with Single Point reported
98.49 % of DGPS (SBAS) positioningmode.
The RTK +Odo RT configuration got
RTK fixed solutions almost all thetime, obtaining the most reliable andaccurate positions.
Qinertia is used to post-process
both
configurations. In open-sky
environment, the odometer does not
improve the situation.

LIGHT URBAN ACQUISITIONDESCRIPTIONThis acquisition is performed in lighturban environment between 15:07
and 15:49 UTC, with low elevationbuildings, bridges, and avenues linedwith trees.

DUT PERFORMANCE
Error analysis compared to reference
An analysis was performed on theentire data to get some quality
indicators. The table below shows
the percentiles at 68 %, 95 % and99.7 % on errors between each DUT
and the reference measurements.
between each DUT and the reference
measurements.

2D position 68%
0.020

2D position 95%

2D posiAltitude
tion 99.7% 68%

Altitude95%

Altitude
99.7%

0.165

0.600

0.057

0.087

0.246

0.520

0.890

1.832

0.813

1.325

1.530

0.009

0.030

0.129

0.007

0.022

0.152

ELLIPSE-D PPK 0.009

0.027

0.069

0.008

0.022

0.097

Yaw 68%

Yaw 95%

Yaw 99.7%

0.089°

0.230°

0.444°

0.096°

0.227°

0.474°

0.067°

0.147°

0.311°

0.054°

0.123°

0.424°

Percentile Error
ELLIPSE-D
RTK+ODOELLIPSE-D
Standalone
ELLIPSE-D
PPK+ODO

Roll /
Roll /
Roll /
Percentile Error Pitch 68% Pitch 95% Pitch
99.7%
ELLIPSE-D
0.017°
0.040°
0.075°
RTK+ODOELLIPSE-D
0.035°
0.079°
0.165°
Standalone
ELLIPSE-D
0.007°
0.014°
0.029°PPK+ODO
0.015°
0.028°ELLIPSE-D PPK 0.007°

Figure 6 - Cumulative distribution error (Light Urban)

CUMULATIVE ERRORS FIGURESThe graphs above show cumulative
distribution errors for 2D Position,
Heading, Pitch and Roll.
In the first graph, we can observethat:» 95 % of Ellipse RTK + Odometer
RT position errors are below 16.5cm.

The real time Ellipse Single Point
reports:» 93.15 % of the time thepositioning mode was DGPS(SBAS).
» 3.28 % of single point due tolimited GNSS reception.
» 2.60 % of ZUPT when the vehicle
stalled.

» 95 % of Ellipse Single Point
position errors are below 89 cm.

» 0.97 % of velocity constraints
when underground or underbridges with no GNSS reception.

From these results we retain thesame conclusion drawn from the previous test. Indeed, even if the GNSS
environment is more challenging, thesolution is totally in line with the SBG
Systems specifications.

The RTK + Odo RT configuration
generates RTK fixed solutions 90%of the time. The odometer usage
allows the system to maintain a goodsolution in case of short period of bad
GNSS reception.

Using Qinertia, results were improvedby post-processing the solution
using SBG Systems Tightly Coupled
IMU+GNSS algorithms. In this case,the GNSS real time algorithms arecompletely bypassed and replaced by
SBG Systems cutting edge INS positioning engine.
We see that both configurations provide very similar results with much
more PPK fixed solutions (89 to 97% for standalone). RTK + Odo setup
reports velocity constraints when the
GNSS reception is too disturbed whereas the Single Point configuration
uses the odometer velocity information to maintain the solution accuracy.

FOREST CANOPY ACQUISITIONDESCRIPTIONThis acquisition is mostly performed
in a forest environment for an hourand half, with trees regularly masking
most of the sky view. This is a very
challenging environment for GNSS
receivers prone to output incorrect
RTK solutions.

DUT PERFORMANCE
Error analysis compared to reference
For this environment, an analysis
was performed on the forest part of
the acquisition only to obtain somequality indicators. As before, thetable on right shows the percentiles
at 68 %, 95 % and 99.7 % on errors
for each sensor and the reference
measurements.

2D position 68%
0.019

2D position 95%

2D posiAltitude
tion 99.7% 68%

Altitude95%

Altitude
99.7%

0.130

0.838

0.062

0.107

1.022

0.620

0.926

1.232

1.574

3.486

4.463

0.011

0.063

0.162

0.011

0.055

0.134

ELLIPSE-D PPK 0.012

0.062

0.493

0.012

0.086

0.470

Yaw 68%

Yaw 95%

Yaw 99.7%

0.073°

0.216°

0.477°

0.093°

0.196°

0.428°

0.053°

0.138°

0.250°

0.049°

0.102°

0.334°

Percentile Error
ELLIPSE-D
RTK+ODOELLIPSE-D
Standalone
ELLIPSE-D
PPK+ODO

Roll /
Roll /
Roll /
Percentile Error Pitch 68% Pitch 95% Pitch
99.7%
ELLIPSE-D
0.016°
0.036°
0.061°
RTK+ODOELLIPSE-D
0.030°
0.067°
0.140°
Standalone
ELLIPSE-D
0.007°
0.015°
0.034°PPK+ODO
0.016°
0.037°ELLIPSE-D PPK 0.007°

Figure 7 - Cumulative distribution error (Forest)

CUMULATIVE ERRORS FIGURESThe graphs above show cumulative
distribution errors for 2D Position,
Altitude, Heading, Roll. Here forexample for the Ellipse Only (Single
Point) in real time, there are about 95% of the heading errors below 0.21°.
From these results we retain that:» The performance of the DUT
matches with specifications.
» The 2D position and altitude
errors are higher than obtained
during the previous tests, due
to poor GNSS reception in thistypical environment.
The real time Ellipse Single Point
reports:» 83.00 % of the time the reported

positioning mode was DGPS(SBAS).

solution using SBG Systems Tightly
Coupled IMU+GNSS algorithms.

» 1.36 % of ZUPT coincides withstop times.

For Ellipse Single Point configuration,
post-processing enables PPK fixed
positions for most of the time (92%) and almost all remaining epochs
where able to get PPK ﬂoat positions
that is quite impressive considering
the environment.

» 15.07 % of single point mode isdue to bad GNSS reception.
» 0.57 % of velocity constraints
when the GNSS reception isgreatly disturbed.
The Ellipse RTK + Odo RT achieves
RTK fixed solutions 90% of the time.
RTK ﬂoat solutions are caused byless reliable solution due to blocked
reception under trees. Odometer wasused when the GNSS signal was too
weak or disturbed to deliver a valid
position.
Thanks to Qinertia, results can be
improved in post-processing the

The Ellipse RTK + Odo RT setup was
already delivering very good results
in real time with almost 90 % of RTK
fixed positions. Qinertia post processing slightly increased fixed solutions
but more importantly was able toimprove the overall accuracy by recovering PPK ﬂoat positions instead ofOdometer integration.

DEEP URBAN ACQUISITIONDESCRIPTIONThis acquisition was mostly performed in harsh urban environment
between 11:44 and 13:12 UTC, withhigh-rise buildings, and tunnels.

DUT PERFORMANCE
Error analysis compared to reference
An analysis was performed on
the entire data to retrieve quality
indicators. The table below shows thepercentiles at 68 %, 95 % and 99.7 %on errors between each DUT and thereference measurements.

2D position 68%
0.096

2D position 95%

2D posiAltitude
tion 99.7% 68%

Altitude95%

Altitude
99.7%

3.479

10.490

0.075

4.314

11.569

1.362

4.337

10.847

4.011

8.011

16.556

0.028

0.299

0.789

0.020

0.172

0.814

ELLIPSE-D PPK 0.030

0.471

1.023

0.018

0.171

0.546

Yaw 68%

Yaw 95%

Yaw 99.7%

0.161°

0.742°

2.245°

0.270°

0.900°

1.749°

0.080°

0.233°

0.399°

0.091°

0.262°

0.701°

Percentile Error
ELLIPSE-D
RTK+ODOELLIPSE-D
Standalone
ELLIPSE-D
PPK+ODO

Roll /
Roll /
Roll /
Percentile Error Pitch 68% Pitch 95% Pitch
99.7%
ELLIPSE-D
0.021°
0.049°
0.089°
RTK+ODOELLIPSE-D
0.037°
0.099°
0.164°
Standalone
ELLIPSE-D
0.009°
0.019°
0.034°PPK+ODO0.020°
0.047°ELLIPSE-D PPK 0.008°

Figure 8 - Cumulative distribution error (Deep Urban)

CUMULATIVE ERRORS FIGURESThe graphs above show cumulative
distribution errors for 2D Position,
Heading, Pitch and Roll.
As expected, both DUT configurations performance are affected bymajor GNSS outages. As for the reference, the RTK + Odo setup reports
RTK Integer status for only 68.11 %of the acquisition, the single point
generates a DGPS status for 70.54 %of the acquisition and a single point
status for 14.17 % of the acquisition.
The post processing improves a lot
the ratio of PPK fixed vs. ﬂoat solutions compared to the real time solution. More importantly, the post processing better rejects invalid GNSS
signals and as a result the accuracy

is greatly improved (10 meters to lessthan 80 cm for 99.7 % percentile).
In Figure 9 representing a harshurban area, the RTK + Odo RT trace(continuous colored line) shows ahuge improvement compared to thepure GNSS solution (yellow dots).

Figure 9 – Comparison between GNSS andINS trajectory in Deep Urban environment

CONCLUSION
The objective of this test report was
to assess the performance of the
Ellipse-D in real life and various environments representative of automotive use cases for a commerciallyavailable INS embedding an industrial grade calibrated IMU. Tests were
performed under open-sky conditions, under tree canopy, and insidevery challenging dense urban environments.
The real time Ellipse standalone and
RTK + Odometer accuracies as well as
the Qinertia post processed solutions
were compared to a post processed
FOG based ground truth (SBG Systems’ Horizon).
The analysis was conducted using
cumulative distribution function(CDF) and percentile distributions toprovide better metrics than “industry
standard” RMS (Root Mean Square)or normal distributions methods.
INS manufacturers often report accuracies in good GNSS conditions orin case of a complete GNSS outage.
However, in real life, even the best
GNSS receiver will report wrong positions with inconsistent accuracies.
This could mislead the INS algorithms and results to unnecessary
GNSS rejections or even solutions
instabilities with bad accuracies.
A trusted INS should cope with thesesituations and deliver the mostreliable and repeatable solution. This
test report demonstrates how correctly and consistently the Ellipse-D
v3 behaves in all observed situations
and more importantly why a calibrated INS is a must have for automotive applications.
Based on the “Ellipse 3, Performance
Assessment Report In Automotive
Application”, we can conclude withthe following statements:» The Ellipse-D v3 demonstrates
adequate capabilities andsufficent accuracies forautomotive applications,» Its unique filtering algorithms
and its dedicated automotive
motion profile lead to obtaining
robust and accurate positioning
and heading results in all driving
conditions,» Its specifications are met witha comfortable margin in all theevaluated conditions,» Odometry improves the solution
accuracy and robustness
especially during long GNSS
outages,» Qinertia Post-Processing extends
coverage for centimeter-level
positioning even in mostchallenging GNSS environments,» Achieved consistent accuracies
makes this small and costeffective Inertial Navigation
System a suitable all-in-one
unit for autonomous vehicles
localization.

We hope that you enjoyed the reading of this assessment report. Please send all your comments and questions
you may have to marketing@sbg-systems.com.

www.sbg-systems.com

SBG Systems EMEA (Headquarters)
Phone: +33 1 80 88 45 00
E-mail: sales@sbg-systems.com

SBG Systems North America
Phone: +1 (657) 845-1771
E-mail: sales.usa@sbg-systems.com