# Survey on Quality of Experience Evaluation for Cloud-Based Interactive Applications
## 5 Strategies for QoE Measurement in CIAs
### Strategies Based on the Instrumentation of Programming Code  
This strategy refers to proposals that require modifying the source code of applications or services running on at least one element of the CIA infrastructure to provide QoE measures. A number of authors tried to evaluate the interactivity time or responsiveness of CIAs through CIA instrumentation. Unlike strategies based on screen updates, authors who opted for code instrumentation did not need to analyse the graphical content of the screen to determine when the response to a user request was represented. Instead, they directly instrumented the CIA code to add timestamps when functions of interest were executed. In this case, the authors aimed to identify when the server received a user interaction and when the client received the graphical response sent by the cloud server. In [35], [PERSON] et al. instrumented code to evaluate QoE in cloud gaming CIAs. They proposed a tool called Pictor for benchmarking cloud gaming services. Pictor has two components. The first one aims to automate and replicate the comparative tests. To do this, Pictor generates interactions with the CIA as a real user would. The second component is responsible for evaluating the behaviour of the service in the face of these emulated interactions. The first component uses neural networks for its operation, so it needs to be trained on labelled recordings from cloud gaming services. Accordingly, the researchers instrumented the cloud gaming CIA client TurboVNC [97] to capture user interactions. Additionally, Pictor uses image-processing techniques to identify the different graphical objects present in the service with which the user can interact. The researcher must label each of these objects during the training phase. The collected information makes up the training set for a recursive neural network, specifically, long short-term memory (LSTM). As a result of training, the neural network can interact with the cloud gaming service in a real scenario. After automating and replicating the interactions of the CIA, the authors subjected the service to different boundary conditions to evaluate its performance. The second component of Pictor allows for comparing configurations and services with each other. Due to the instrumentation, Pictor can obtain the interactivity time. Specifically, the authors instrumented the service to monitor calls to several common graphics library functions (OpenGL [98]), allowing them to determine how long the CIA takes to render graphics objects after an interaction. [PERSON] et al. [36] sought to measure interactivity time by instrumenting the HTML and JavaScript code of interactive web applications. When the user interacts with an interactive web application, it triggers the rendering of various HTML components. This approach requires a taxonomy of all HTML elements on the web. In a previous work, the researchersdetermined how many elements rendered after a given interaction represented the final graphical state of the associated response. For each possible interaction with the CIA, the authors calculated the thresholds for the triggered events. Using this instrumentation, the authors monitored user input and the number of HTML elements rendered. When the CIA receives an interaction, the instrumented code calculates the moment at which the interactive web application renders all HTML elements of the associated response. The output time metric is an approximation of the interactivity time, as it does not account for the time required by the browser to transfer the screen refresh to the operating system. In the field of interactive web applications, the literature presents various alternative time metrics to interactivity time achieved through the instrumentation of their code to represent QoE. [PERSON] et al. [40] presented a tool called Web View, which automates the collection of several of these metrics.