# From Quality of Experience to Willingness to Pay for Interconnection Service Quality

Andreas Sackl, Patrick Zwickl, Peter Reichl

► To cite this version:

Andreas Sackl, Patrick Zwickl, Peter Reichl. From Quality of Experience to Willingness to Pay for Interconnection Service Quality. Networking Workshops (NETWORKING), May 2012, Prague, Czech Republic. pp.89-96, 10.1007/978-3-642-30039-4_11. hal-01533590

HAL Id: hal-01533590

https://inria.hal.science/hal-01533590

Submitted on 6 Jun 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# From Quality of Experience to Willingness to Pay for Interconnection Service Quality*

Andreas Sackl, Patrick Zwickl, and Peter Reichl

FTW Telecommunications Research Center Vienna,
Tech Gate Vienna, Donau-City-Straße 1/3rd floor, 1220 Vienna, Austria
{sackl,zwickl,reichl}@ftw.at
http://www.ftw.at

**Abstract.** Triggered by the pending success of Quality of Service (QoS) differentiation in practice, recently the research interest has been increasingly focussing on the user-centric marketization of QoS, especially for interdomain scenarios. In this context, the key role of readiness and willingness to pay for enhanced network quality has not sufficiently been covered so far. In this paper, we focus on the users' willingness to pay for realtime network quality in interactive Video-on-Demand scenarios from an empirical perspective. Our user trial results indicate a broad willingness to pay for enhanced network quality, as well as remarkable influences on the quality perception through purchasing decisions, which is expected to kick off further experiments and the adaption of existing Quality of Experience models.

**Keywords:** Quality of Service, Quality of Experience, Willingness to Pay, Video on Demand

## 1 Introduction

Quality of Service (QoS) differentiation for network services has been the center of many discussions in the past and presence, recently also in the interconnection (IC) context for providing quality guarantees beyond domain borders, e.g., in the ETICS project<fnref n="1" />. Especially in times of rapid network traffic growth rates in IC opposed to stagnating revenues, there is an increasing conviction that well configured QoS differentiation mechanisms may be economically advocated.

In practice however, the strong inherent needs of relating QoS to economics and charging policies – as often stated in literature (e.g., [1]) – and of aligning

<footnote>* The authors would like to thank Sebastian Egger and Raimund Schatz for their support and fruitful discussions. The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement n°248567 for the ETICS project. Further information is available at www.ict-etics.eu. FTW is funded within the COMET Program by the Austrian Government and the City of Vienna.</footnote>

<footnote n="1">¹ https://www.ict-etics.eu/</footnote>
QoS differentiation mechanisms to customer demands, i.e., Quality of Experi-
ence, have been insufficiently satisfied by the available technical solutions, which
have lead to the debatable success of network QoS so far [2].

This is further hampered by the diverse definitions and models available for QoE which have rather focused on integrating the user's perspective in the overall network environment [3-8] or quantifying the relationship between audio and video qualities and QoE [7, 9, 10]. However, these works have often failed to link the economic aspects in the users perceptions to concrete terms like the willingness to pay for certain QoS levels. Hence, this work contributes to fill this gap by investigating the end customers willingness to pay for improved network transmission quality by means of a user trial. According to [11], willingness to pay quantifies the amount of money to be spent for a given quality, while the related concept of readiness to pay may be defined as the general disposition to pay for a given quality. Positioned between these two bounds, our empirical study investigates how much end customers pay for Video-on-Demand (VoD) streaming quality in order to better integrate end customers interest in quality differentiations.

Related work on this topic is rather limited. Probably closest to our work, the study [12] has initially linked the idea of different quality opportunities with consumers’s monetary decisions. In contrast to our approach, [12] focused on pre-rendered video qualities, i.e., videos were presented with different bitrates. The monetary decisions were analyzed by randomly assigning users to user profiles with varying prices for predefined video bitrates. Within this setup, the M3I experiments demonstrated users willingness to pay for higher QoS. Based on that, our research intends at transferring these monetary aspects of video qualities to realtime transmission interconnection scenarios, i.e., a common VoD-scenario, where QoS is modeled in terms of different packet loss rates and linked to a certain pricing structure, in order to identify implications on the technical management of network QoS classes and their marketization in terms of sellable goods.

The contribution of the present work is therefore twofold: on the one hand we investigate the willingness to pay for improved network quality in a realistic real time video streaming scenario, and on the other hand we incorporate Quality of Experience aspects like acceptance rates and subjective video quality perceptions.

The rest of this paper is structured as follows: we start with describing in detail the technical setup and the procedure of the user trial. Section 3 discusses the results of our empirical study including user demographics, user behavior and of course outcomes regarding willingness to pay. Finally, Section 4 presents our conclusions.
## 2 Experimental setup

In order to provide for an utmost realistic live streaming VoD scenario, we have taken recourse to FTW's iLab user test laboratory<fnref n="2" /> for creating the atmosphere of a living room-like situation. Based on different levels of random packet loss rates, after extensive pre-tests, we have formulated four relevant network quality levels (i.e., price and packet loss percentage – see Table 1). This setup is in addition backed by an extensive library of modern movies, TV series, and documentations, as well as by realtime UDP network transmission of the movies with a realistic constant delay of 75 ms and a variable packet loss on the used link.

<table><caption>**Table 1.** The offered quality classes</caption><thead><tr><th></th><th>Quality 1</th><th>Quality 2</th><th>Quality 3</th><th>Quality 4</th></tr></thead><tbody><tr><td>Packet loss [%]</td><td>1</td><td>0.25</td><td>0.085</td><td>0</td></tr><tr><td>Delay [ms]</td><td>75</td><td>75</td><td>75</td><td>75</td></tr><tr><td>Costs per min [€]</td><td>0</td><td>0.025</td><td>0.05</td><td>0.075</td></tr><tr><td>Costs per movie [€]</td><td>0</td><td>0.5</td><td>1</td><td>1.5</td></tr></tbody></table>

*Procedure.* Inspired by [12], our experiment creates an interactive quality market allowing the consumers to purchase realtime quality enhancements of the streamed video with real money. For this purpose, each user is assigned a balance with an initial deposit of €10 for their free disposal in the experiment and/or afterwards. Thereafter, the users individually watch three videos of their choice with a duration of 20 minutes each, which are automatically launched in the worst available quality. Each movie starts with an initial trial phase (2-3 minutes duration, free of charge) for experimenting with the different quality levels and eventually deciding (and purchasing) one quality level which is provided afterwards. During the video runtime, this pattern is repeated three more times, i.e., trial phases for upgrading the original decision are also offered after 5, 10 and 15 mins (note that no downgrading is allowed throughout). At the end of the experiment the remaining deposit (Euro) is paid out in cash to the users hence capturing the willingness to invest the own money for network quality upgrades.

*Laboratory.* From the technical perspective, in our setting the video stream is triggered by an *iPad* (cf. Figure 1). The test users can select a video of their interest from our extensive collection. By choosing a video, the iPad calls a script

<footnote n="2">² FTW Interfaces & Interaction Lab (i:lab): http://www.ftw.at/portfolio/i-lab?set_language=en, last accessed at Jan 26, 2012</footnote>
which starts the VLC<fnref n="3" /> video stream on our *Linux server*. At the same time, a packet loss script randomly dropping packages at a given percentage is initiated through a netem<fnref n="4" /> command providing network emulation capabilities for Linux systems. This sets up the defined network quality for the starting phase – the worst quality offered in our experiment. The video is then streamed via the network to a *Mac Mini Server*, which displays the received videos on a directly connected *flat screen television*. The Mac Mini in this case appears to the user as set-top-box, hiding the details of the transmission logic, i.e., network details and server structures. The iPad, moreover, acts as remote control offering the users to purchase quality upgrades.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Technical setup of the experiment</figcaption></figure>

Our technical setup also foresees the capability of constantly increasing the packet loss between two *trial phases* without explicitly notifying the test users. This process is applied between the first and the second trial phase of the third movie. The packet loss is continuously increased by 0.2% in a modification interval of 60 seconds, while leaving the price calculation stationary. The resulting packet loss deterioration then remains active for the residual part of this movie and is cleared afterwards again. With these data, we intend to gain further understanding on relevant triggers for the users’ quality decisions.

## 3 Results

Based on the concept described above, a laboratory-based user trial was scheduled in Vienna in October 2011. Its results have been intensively statistically analyzed and are presented in this section.

<footnote n="3">³ VideoLAN Client (VLC): http://www.videolan.org, last accessed at Jan 26, 2012</footnote>

<footnote n="4">⁴ Netem: http://www.linuxfoundation.org/collaborate/workgroups/networking/netem, last accessed at Jan 24, 2012</footnote>
## 3.1 Test User Demographics and Background

Overall, 43 users (22 male and 21 female) successfully participated in our study. Approximately 40% were between 18 and 29 years old, 32% were between 30 and 44 years old and 28% were older than 45 – the mean age was 36.8. Most of our users where employed (48%) or students (28%). More than 93% of the test users were familiar with YouTube, 75% of them used this service at least once a week. Most of them have consumed music videos (67%), while movies and fun videos have been of limited interest (20%). On the other hand, only 5 users have consumed videos from dedicated VoD platforms (2x iTunes, 2x A1 Videostore, 1x UPC on demand) with a mean monthly spending of €5.48 before.

## 3.2 Willingness to Pay

Figure 2 depicts each user's remaining deposit after the consumption of the three videos. A subset of 9 "generous" users have spent the maximum amount of money (€4.5 of their €10 balance i.e., €1.5 per movie), while 4 "budget-minded" users decided to watch all three movies in the worst quality (and receiving the maximal payout of €10 in cash). The majority, however, has taken an intermediary position between these two extremes.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Users’ remaining deposits after three rounds</figcaption></figure>

The average of all 43 users has spent €1.01 (±0.49, standard derivation) per movie to increase the quality, i.e., to decrease the packet loss of the transmission – corresponding to two-thirds of the maximum potential spending of €1.5. Consequently, this provides a proof for the feasibility of our setup linking real cash payouts with a laboratory setup. The spent money is obviously subject to the presented quality. The mean chosen quality level per movie was 3.04 (±0.99),
i.e., on average the second best quality level was chosen. Most purchases have been made during the first trial phase of each movie. During the rest of the movie, only limited quality modifications have been applied. Not even continuously decreasing prices – as calculated on the basis of the remaining share of the movie – have triggered a higher intensity quality upgrades.

## 3.3 User Behaviors

The observed behavior of our participants provided a revealing insight in the chosen purchasing and consumption strategies:

- **Strategic players:** Some users have repeatedly chosen the best quality during the trial phases (free of charge). However before the end of each trial phase, they have returned to the poorest quality level again – i.e., no quality purchases have been made in order to receive the maximum deposit payout.

- **Generous players:** We could observe users always selecting the best quality at the beginning of the movie without testing lower qualities. Thereafter, they have stated their insensitivity to payments from their deposit and their intention to watch all movies in the best available quality.

- **Budget-minded players:** Some users have declared after the trial that they would not care about the quality as long as they would receive the full €10 payout. In some cases, better quality has not even been tested.

- **Quality & price-aware players:** Most of the users have tested all quality levels at the beginning and have finally chosen quality 3 or 4 (willing to pay the corresponding charges).

Generally, most users chose the best or the second best quality in the first trial phase without extensively testing the available options. The mean click rate during the first trial phase of each movie is 4.17 clicks, i.e., every user has only changed the quality at the beginning four times on average. During the rest of the movie, the click-rate has been even lower (mean click rate is 6.5): the majority of users changed the quality less than 10 times per movie. The average click-rate per user remained constant over time respectively over the three selected videos.

## 3.4 Acceptance

Another interesting question concerns the relationship of perceived quality and user acceptance. Related work has e.g., studied the correlation between MOS ratings and acceptance, for example [13] examined this correlation for mobile broadband usage. There are also studies (e.g., [14]) that have examined the influence of codec settings, content types, various devices etc. on the acceptance. The analysis of different video qualities regarding MOS ratings and acceptability, however, to the best of our knowledge has not been addressed by the research community. Therefore, we have asked the test persons also to fill in a questionnaire on their perceived video quality after each video, i.e., ACR 5 MOS-scale [15], and whether they would chose such a quality at home, i.e., a video acceptance rate. To our surprise, even lower video quality levels are acceptable
for the majority of the users (Figure 3). These findings are currently object of
investigation and further studies are planned to analyze this attitude.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Users’ acceptance per qualityclass (packetloss). 1=worst quality; 2=best quality</figcaption></figure>

**3.5 Further findings**

There were no significant statistical correlations between the click-rate, the spent money or the chosen quality level and age, sex, education or YouTube usage of the participants. During the third movie, a hidden packet loss boost (+0.2%) has been initiated after the first trial phase for all quality levels lower than 4 (31 of 43 test users did not choose the best quality level). While 21 users have not reacted to this hidden deterioration, 9 participants increased the quality by one level and one user increased it by two levels – the packet loss boost has only affected about one third of all eligible users.

**4 Conclusions**

Our analysis has demonstrated the feasibility of the used laboratory setup for re-
alistic willingness to pay studies for VoD scenarios. This is also confirmed by the
existence of a substantial willingness to spend money for enhancing the network
quality, i.e., reducing network packet loss. Consequently, our work may serve
as basis for more complex investigations regarding e.g., users' price elasticity to
quality changes.

One particular phenomenon could be observed regarding the users interaction
habits with the quality market. After their initial choice, a strong tendency of
rarely modifying chosen quality levels has often educed the total ignorance of
three out of four upgrade opportunities. We could also show the presence of
various interesting purchasing strategies such as users continuously purchasing
the highest quality or the lowest price option – anticipating four types of players.

Due to the limited user reactions (33%) on a hidden packet loss boost (+0.2%), we may argue that pricing has been predominately the decisive factor
for purchasing decisions after the initial quality level choice. Like with a couple
of other issues already mentioned earlier, further clarification on this point will
be subject of future work.

References

1. Jain, R.: Internet 3.0: Ten problems with current Internet architecture and solutions for the next generation. In: Proceedings of the Military Communications Conference (MILCOM'06), IEEE (2006) 1-9

2. Burgstahler, L., Dolzer, K., Hauser, C., Jähnert, J., Junghans, S., Macián, C., Payer, W.: Beyond technology: the missing pieces for QoS success. In: Proceedings of the ACM SIGCOMM workshop on Revisiting IP QoS (RIPQoS '03), ACM (2003)

3. Kilkki, K.: Quality of Experience in Communications Ecosystems. Journal of Universal Computer Science, Special issue on SocioEconomic Aspects of Next Generation Internet (2008) 615–624

4. International Telecommunication Union (ITU): Vocabulary for performance and quality of service, amendment 2: new definitions for inclusion in Recommendation P.10/G.100. (2008)

5. Lopez, D., Gonzalez, F., Bellido, L., Alonso, A.: Adaptive multimedia streaming over IP based on customer oriented metrics. In: International Symposium on Computer Networks. (2006)

6. Soldani, D., Li, M., Cuny, R.: QoS and QoE Management in UMTS Cellular Systems. Wiley (2006)

7. Reichl, P., Tuffin, B., Schatz, R.: Logarithmic Laws in Service Quality Perception: Where Microeconomics Meets Psychophysics and Quality of Experience. Telecommunication Systems (2011)

8. Fiedler, M., Kilkki, K., Reichl, P.: From Quality of Service to Quality of Experience. In: Proceedings of the Dagsuhl Seminar 09192. (2009)

9. Staelens, N., Moens, S., Van den Broeck, W., Mariën, I., Vermeulen, B., Lambert, P., Van de Walle, R., Demeester, P.: Assessing Quality of Experience of IPTV and Video on Demand Services in RealLife Environments. IEEE Transactions on Broadcasting 56(4) (2010) 458–466

10. Zhongkang, L., Weis, L., Boon Choong, S., Kato, S., Eeping, O., Susu, Y.: Perceptual Quality Evaluation on Periodic FrameDropping Video. In: Proceedings of the International Conference on Image Processing, IEEE (2007)

11. Ries, M., Nemethova, O., Rupp, M.: On the Willingness to Pay in Relation to Delivered Quality of Mobile Video Streaming. In: Proceedings of the International Conference on Consumer Electronics, IEEE (2008)

12. Hands, D., ed.: FP5 Project M3I, IST199911429, Deliverable 15/2 – M3I user experiment results. (2002)

13. Schatz, R., Egger, S., Platzer, A.: Poor, Good Enough or Even Better? Bridging the Gap between Acceptability and QoE of Mobile Broadband Data Services. In: Proceedings International Conference on Communications (ICC'11), IEEE (2011)

14. Knoche, H., McCarthy, J., Sasse, A.: Can small be beautiful?: assessing image resolution requirements for mobile TV. In: Proceedings of the 13th International Conference on Multimedia. (2005)

15. International Telecommunication Union (ITU): ITU-R BT.500-7 Methodology for the subjective assessment of the quality of television pictures. (2002)