QoS Made Easy
Smart Measurement and Metrics

A good user experience?
When smartphone users surf the Internet, watch YouTube™ videos, listen tomusic on Spotify™, or just make phone
calls, they expect it to work without
problems.

However, if loading a page takes forever,if video streaming freezes, or if there arelong delays when making a call or thecall is aborted, the user experience suffers.
The problem: Mobile customers usuallydon‘t know and don‘t care if the cause
is a bad mobile connection—and theproblem therefore in the realm of theprovider—or if the server at the otherend is overloaded because many users
try to access the same web resource(website, video, etc.). What counts
for them is the end-to-end experience.

Measurable parameters (KPI)
ceerien ters
pxer
seme
The u ed in para S..
sres
or Qo
is exp d KPI, KQI
calle

Eachservice(such astelephony, webbrowsin
or videogstreaming) has
its own
KPI.

With Voice-over‑IP,
telephony also uses
standard data transmission
protocols.

Data
VoIP ty packets ofcbe giv pes, such a ertainen pres VoLT
E, cfeto imp rential trea antment
rove quality.

Mobile radio technology generations
Mobile radio access technologies arecategorized by generations: 2G, 3G, 4G,
5G etc. Each main generation has multiple subtechnologies that are the resultof evolving main technologies.

The most obvious difference between
these technologies is the maximum
achievable data rate.

Mobile radio generations, technologies and services
2G

3G

4G

5G

Standards

GSM, CDMA,EDGE, GPRS

UMTS,CDMA2000,
HSDPA, EVDO

LTE Advanced,WiMax

3GPP Rel. 15

Voice transmissioncircuit switchedcircuit switchedpacket switchedpacket switched

Data transmission

circuit switchedpacket switchedpacket switchedpacket switched

Data rate

up to 473 kbit/s

up to 42,2 Mbit/s

up to 1 Gbit/s

up to 7,5 Gbit/s

Services

text messaging,text-based email

web browsing,email, audio andvideo streaming

HQ video
streaming, VoIP

virtual andaugmented reality(VR/AR), Internetof Things, autonomous driving etc.

The maximum data rate
The achievable maximum data rate is influenced by various factors:■ The available radio access technology■ The ramp-up phase, i.e. the time until the maximum data rate is reached. As aconsequence, the maximum data rate is not reached with small data volumes■ The number of concurrent users in a mobile radio cell
■ Signal level and connection quality

Together, these factors determine
what sort of quality users can expect
when using a particular type ofservice on their mobile devices.
They also give an indication whether
an increase in technical performanceleads to a perceptible increase inquality at all.

This means:
A service used in a stable 3G network may result in a better user
experience than the same service
used in a less stable 4G network.

Service type and throughput
Data throughput is defined as the amountof data transmitted per time unit (kbit/s,Mbit/s). Different services have different
demands on the data rate that is necessary for smooth operation.
The table on the right shows some typicalvalues for various services.
However, these values can only give arough indication of the throughput
values necessary for a good user experience for a given service.

ServiceData Rate

Video streaming

1 - 10 Mbit/s

Web browsing100-200 kbit/s

Music streaming

100 kbit/s

IP Telephony30-100 kbit/s

Video Telephony

1 Mbit/s

Text Messaging

1-10 kbit/s

Mail

100 kbit/s

Data test KPI
Data service tests can be categorized as follows:

Data Test Type

Upload/DownloadService Characteristics

Testing the maximum
achievable data rate

Web browsing

Many small elements areloaded partly in parallel,
partly one after the otherVideo streaming

For video streaming, aconstant minimum data
rate is more important
than temporary peaksKPI■Throughput■Connection failure rate■

Duration of data transfer■

Drop rate■

Loading time■

Drop rate■

Frequency of „freezing“■

Drop rate■

Time to playback

Telephony KPI
A typical test scenario for telephony involves
two devices, in which the first device calls thesecond one. Once the connection is successfully established (the call setup time is registered),the connection is held for the predefined time
and then terminated.
The most important KPIs for telephony are:■ Voice quality, measured, for example,
using the internationally standardized
ITU-T P.863 (“POLQA“) method■ Stability of the connection: how long
can the connection be maintained?
■ Delay between speaking andlistening – also called latency ordelay – especially for over-the-top VoIP
services like Skype™ or WhatsApp™

Data packets of VoIP services
that are part of the standardized network technology (e.g.
VoLTE for 4G), are prioritized.
Voice data packets of over-thetop (OTT) services, however,are regarded as standard datapackets. The probability ofdelays is therefore higher.
If a delay is longer than approx.
½ second, participants might
unintentionally interrupt eachother, because they assume
the other party is making apause. This can result in a very
poor quality of experience.

QoS is business value
User experience is a key success factor
for mobile network providers andindustries whose business model largely
depends on a well-performing mobilenetwork (for example, music or video
streaming services).
For these stake­holders, it is therefore
important to obtain meaningful datathat shows where investment in themobile network (or other components)
is necessary to optimize the user
experience and guarantee the service
quality promised to customers.
CustomerGoals

If companies invest with the objective
to increase the performance of themobile network, this data provides
information how investment can bemade with the best cost-benefit ratio.
Ideally, a set of well-chosen QoS performance indicators describes thecustomer‘s benefits and therefore thebusiness value of a mobile network.
Additionally, this data serves as a basis
for determining how services are communicated to the customer or what afair use policy would have to look like.

FI Consultancy
Services

FI Measurement
Services

FI Consultancy
Services

Translate question
into methodology

Develop measurement
concept and carry out
measurement campaigns

Convert raw
data into usefulmeasurement
results

Customer
Decisions

Measuring the user experience
At this point, it is important tokeep in mind that the quality
as perceived by a user is influenced by factors that network
operators cannot completelycontrol. As a mere transport
medium, the mobile network
itself is not to blame if for example the server that delivers
a website or video is overloaded.
However, the average user isgenerally not interested in theroot causes of poor performance. A network operator will
therefore use end-to-end
measurements with realistic
use cases combined with diagnostic tests to identify theorigin of problems.

In order to measure the user experience, itis necessary to consider the entire utilization
action. For a telephony service, for example,
this includes the complete process fromestablishing a connection after the user hasentered a phone number until the end ofthe call.
Combinations of end-to-end tests anddiagnostic procedures are also used by otherstakeholders, e.g. companies that purchasemobile services from network operators orcorporate customers with devices foremployees. KPI and targets for such services
are defined in Service Level Agreements(SLA).

Our measurement systems
The whole range from smartphone-only to multi-channel measurement
All Focus Infocom measurement systems master end-to-end service tests withstandardized KPI.
Where KPI do not exist or where they are not sufficient, we will develop suitable parameters with the aim of describing the user experience as realisticallyas possible. This can also be done in close cooperation with our clients forindividual use cases.
With our unique SAM (Smart App Manager) technology, we also provide aplatform that allows us to control, measure and evaluate the activities ofsmartphone apps such as Facebook™, WhatsApp™, Spotify™, or YouTube™,
using KPI that can be freely defined.

Testcase in progress…

Next upload due in 15 min.

Get in touch!
Focus Infocom
Gesellschaft für Informatik undTelekommunikation mbH
Heinrichstraße 2D-64283 Darmstadt
Germany

Phone: +49 6151 971100
Email: sales@focus-infocom.deInternet: https://focus-infocom.de

All data in this booklet anonymized for confidentiality reasons.

Focus Infocom is a Member of ETSI and Associate Member in ITU-T SG12