Logging encrypted data communications for QOE analysis

A device such as a smartphone may communicate with a server or other network entity using encrypted communications, making it difficult to examine such communications for purposes of identifying communication issues that may affect user QoE (quality of experience). In certain embodiments, an application may be modified to log communication data before encryption and after decryption. For example, the application program may be decompiled and logging instructions may be inserted before portion that result in data encryption and after portions where received data is decrypted. The modified application program may then be recompiled and executed on a device to produce an unencrypted log of data. In other embodiments, elements of the device operating system may be modified to log data before encryption and after decryption.

BACKGROUND

Applications that are installed and/or running on mobile devices often rely upon the services of websites and associated servers that are accessible using wireless network communications. Because of this reliance, application performance and quality of user experience (QoE) may be affected by the ways in which the websites and servers provide data to the applications.

In many situations, it may be possible to monitor data communications between an application and a server and to analyze the communicated data in order to identify things that may adversely affect application and/or device QoE. In some cases, for example, an analyst may determine that server-provided responses to certain requests are delayed, causing an application to seem “laggy” to a user. In other cases, a server may return faulty data that causes repeated requests or other problems when received by the mobile device. In yet other cases, examination of a data flow may reveal problems with network communications paths between the mobile device and a server.

In certain situations, a mobile device may communicate with a server using encrypted communications. For example, a communication protocol known as HTTPS (hypertext transfer protocol secure) may be used in order to ensure privacy and to authenticate data sources. When using a secure communication protocol such as HTTPS, it is difficult or impossible for an analyst to decipher communications between a mobile device and a server, effectively eliminating an analytical tool that would otherwise be available for improving the performance of an application.

DETAILED DESCRIPTION

Described herein are techniques for evaluating issues relating to user quality of experience (QoE) in systems that use encrypted network communications. In an example system, an application program runs on a mobile device under a mobile device operating system. The mobile device and its operating system provide functionality for communicating with remote servers through a wide-area network such as the Internet. In some cases, this functionality is implemented through a network communications protocol stack having a series of protocol layers. For example, a TCP/IP (transmission control protocol/Internet protocol) stack may include an encryption/decryption layer that encrypts outbound data prior to sending the data to a server and that decrypts inbound data after receiving the data from the server. The application program communicates with the server through the network communications protocol stack so that both outbound and inbound data are encrypted.

In one example embodiment, the application program may be decompiled and modified to log unencrypted outbound data before the outbound data is submitted to and encrypted by the network communications protocol stack of the mobile device operating system. The application program may also be modified to log inbound data after it has been received from the network communications protocol stack and decrypted by the mobile device operating system. For example, in order to analyze QoE of an application program written and compiled using the Java programming language, the application program may be decompiled to create a Java bytecode listing of the application. The bytecode listing may then be analyzed to identify points within the program at which data is sent to or received from the network communications protocol stack, and logging functionality inserted at these points so that unencrypted HTTP (hypertext transfer protocol) data is written to a log file in the file system of the mobile device. The bytecode listing is then recompiled, installed on the mobile device, and executed. The resulting log file can then be accessed and analyzed to identify problems or delays in data flows.

In another example embodiment, the network communication protocol stack itself is modified to log network communication data before encryption and after decryption. For example, the TCP/IP (transmission control protocol/Internet protocol) network communications protocol stack of an open-source operating system such as the Android mobile device operating system may be modified to log outbound data prior to encryption and to log inbound data after decryption. Unencrypted HTTP data, for example, may be written to a log file on the mobile device and later accessed by an analyst to identify problems or delays in HTTP data flows.

FIG. 1illustrates relevant software component of an example mobile device102that is configured to communicate through a wide-area network (WAN)104with a server106. The mobile device102may comprise a telecommunications device such as a smartphone, which provides wireless network communications through a cellular telecommunications network with the Internet and with websites and servers that are accessible through the Internet. In the example ofFIG. 1, the server106may be accessible through a public network such as the Internet or through a private network associated with a telecommunication provider that supports the mobile device102. For example, the mobile device102may communicate with certain servers that are accessible only to mobile devices supported by the telecommunication provider. Servers such as this may provide, for example, billing and other account information relating to usage of the mobile device102on the cellular communications network of the communication provider.

In other embodiments, the mobile device102may comprise a tablet computer, a laptop computer, a wearable device, a media player, or any other type of device that implements a form of encrypted data communications with remote servers, data sources, or other entities. Furthermore, the described techniques may be implemented in devices other than mobile devices, such as desktop computers, network-enabled appliances, home security systems, home automation systems, other types of communication systems, industrial control systems, automotive computers, and various other types of devices and systems that utilize encrypted data communications.

An application program108is installed on and executed by the mobile device102. The application program108may be any type of software that communicates with and accesses data or services of the server106. In some cases, the application program108may be an integrated system component, such as an Internet browser or other component that is a preconfigured part of the mobile device102or of the device's operating system. In other cases, the application program108may be a third-party or after-market software component that is purchased, downloaded, or otherwise obtained by a user of the mobile device102and installed on the mobile device102after purchase of the mobile device102.

The mobile device102has a network communication protocol stack110, which may be part of the operating system of the mobile device102. The network communications protocol stack110has multiple protocol layers112, each of which may be a software component configured to implement a corresponding protocol. Outbound data sent by the application program108to the server106is processed in turn by each of the protocol layers112, from top to bottom. Inbound data received from the server106is similarly processed by the protocol layers from bottom to top of the illustrated stack110.

In use, the application program108provides HTTP-formatted paths and query strings to the network communications protocol stack110. The paths and query strings are encrypted by the network communications protocol stack110prior to transmission to the server106. The application program108also receives HTTP-formatted responses and other data from the server106. The network communications protocol stack110decrypts such responses and data before providing them to the application program108.

At the top level of the network communications protocol stack110, closest to the application program108, is an encryption/decryption layer112(a). In an HTTPS environment, this may comprise a TLS (transport layer security) component or an SSL (secure sockets layer) component that encrypts outbound data and decrypts inbound data using public key cryptography. When using an encryption/decryption layer such as this, all HTTP headers, URLs (uniform resource locators) and message bodies are encrypted for transmission between the mobile device102and the server106.

At the next lower level of the network communications protocol stack110is a transport layer112(b). In many embodiments the transport layer112(b) may comprise a TCP (transmission control protocol) protocol layer.

Beneath the transport layer112(b) is a network protocol layer112(c), which may comprise an IP (Internet protocol) protocol layer.

Beneath the network protocol layer112(c), closest to the server106, is a data link layer112(d), which may implement an Ethernet protocol or other LAN (local-area network) protocol.

Note that the network communications protocol stack110shown inFIG. 1is simplified, and may include other layers that are not shown.

A logging layer or component114may be logically positioned between the application program108and the encryption/decryption layer112(a) of the network communications protocol stack110, so that both outbound and inbound data are processed by the logging component114. The logging component114logs all outbound and inbound HTTP data to a log file or other data structure116on the mobile device102. Because the logging component114is between the application program108and the encryption/decryption layer112(a), the HTTP data at this point is not encrypted. Accordingly, the log file116contains unencrypted HTTP data, including HTTP strings submitted by the application program108and HTTP strings received from the server106. The logged HTTP data may include all requests and responses, including headers, URLs, message bodies, and so forth, and may also include metadata such as the time at which each of these items was transmitted or received.

In practice, the logging component114may be implemented in various ways. In one embodiment, for example, the logging component114may be implemented within the application program108. In another embodiment, the logging component may be implemented by the network communications protocol stack110as an additional protocol layer. In yet another embodiment, the logging component114may be implemented by the encryption/decryption protocol layer112(a) of the network communications protocol stack110, in a way such that the encryption/decryption protocol layer112(a) logs unencrypted HTTP data, before encryption and after decryption.

FIG. 2illustrates an example method200for evaluating quality of experience (QoE) provided by a mobile device102or other device that communicates with and utilizes services of a server using an encrypted communication protocol. The method200may be performed, as an example, in order to evaluate the QoE provided by an application program108that is configured to run on the mobile device102.

An action202comprises obtaining a software component that is configured to run on the mobile device102, wherein the software component is configured to communicate with the server using an encrypted communication protocol such as TLS or SSL. As one example, the software component may comprise an application program108that is designed and configured to run on the mobile device102or other type of device and to communicate with a server using encrypted communications. The application program108may itself perform encryption or may use the services of a device operating system to perform encryption. As another example, the software component may comprise a device operating system or a communication component of a device operating system, such as a network communication protocol stack110of a device operating system. In some implementations, the software component may comprise a TCP/IP network communication protocol stack.

An action204comprises modifying the software component to log outbound data before encryption of the outbound data and to log inbound data after decryption of the inbound data. The action204may comprise analyzing the software component to locate portions of the software component that cause encryption or that initiate sending of encrypted data. The action204may also comprise analyzing the software component to locate portions of the software component that cause decryption of inbound data or that receive the decrypted data resulting from decryption. The modifying204may comprise inserting instructions at the located portions of the software component to write unencrypted data to a log file or other data store that is in the file system of the mobile device102.

An action206comprises installing the modified software component on the mobile device102. For example, the action206may comprise installing a modified application program on the mobile device102. As another example, the action206may comprise installing a modified operating system or network communication protocol stack on the device102. In some situations, installation and/or execution of the modified software component may require elevated user privileges, which may be obtained by “rooting” the device to gain administrative privileges.

An action208comprises running the modified software component on the mobile device102. For example, the action208may comprise running the modified application program on the mobile device102, whereupon the modified application program will write unencrypted inbound and outbound data to a log file on the device. As another example, the action208may comprise running the modified operating system or network communication protocol stack, whereupon the modified protocol stack will write unencrypted inbound and outbound data to a log file on the device.

An action210comprises accessing the logged data. For example, the mobile device102may be connected to a computer and the logged data may be transferred to the computer for analysis. As another example, the mobile device102may be configured to send the logged data to a server or other computer for analysis.

An action212comprises analyzing the inbound and outbound logged data to evaluate QoE provided by the mobile device102and/or an application running on the mobile device102. The action212may in some cases be performed by a human analyst, by examining one or more log files created by the modified software component. In some cases, the logged data may comprise HTTP queries and responses, and the action212may comprise analyzing the HTTP queries and responses to identify sequences that may negatively affect user QoE, such as delayed queries or responses, incorrect queries or responses, etc.

FIG. 3illustrates a more specific example method300for evaluating QoE provided by a mobile device102that communicates with a server106using an encrypted communication protocol. The method300may be performed, as an example, in order to evaluate the QoE provided by an application program108that is configured to run on the mobile device102and to exchange encrypted data with the server106.

An action302comprises decompiling the application program108to produce a decompiled application304. In certain situations, the application program108may comprise an application written and compiled using the Java programming language and a Java compiler. The action302may comprise using a Java decompiler to produce the decompiled application304based on the binary class files of the application program108, where the decompiled application304may comprise a byte code or source code listing of the Java application program108.

An action306comprises inspecting the source code of the decompiled application304to identify any portions or points within the application that cause transmission and/or encryption of outbound data. In many cases, the action306may comprise locating instructions or functions within the decompiled application304that receive outbound data prior to encryption of the outbound data and/or that initiate sending of outbound data to the server106. For example, such instructions may be ones that call an API of the mobile device operating system, providing HTTP-formatted data that is to be encrypted and sent to the server using the network communications protocol stack110.

An action308comprises modifying the identified portions of the application and thereby configuring the application program to log outbound data prior to encryption of the outbound data. This may comprise inserting one or more instructions into the decompiled application304for execution prior to the identified portions or points of the application that cause transmission and/or encryption of the outbound data. The inserted instructions, when executed, cause the mobile device102to write the unencrypted outbound data to the log file116prior to encryption of the outbound data. In the example described herein, the inserted instructions may comprise Java instructions that call functions or APIs provided by the device operating system to write to the file system of the mobile device102. The logged data may include outgoing HTTP strings as well as metadata associated with the strings such as timestamps.

An action310comprises inspecting the source code of the decompiled application304to identify any portions of or points within the application that receive inbound data after decryption of the inbound data. For example, such an instruction may comprise one that calls or is called by an API of the mobile device operating system to receive inbound data that has been received from the server106using the network communications protocol stack110.

An action312comprises modifying the identified portion of the application, and configuring the application program to log the inbound data after the decryption of the inbound data. This may comprise inserting one or more instructions into the decompiled application304for execution after the portions of the application identified in the action310. The inserted instructions, when executed, cause the application program108and mobile device102to write the decrypted inbound data to the log file116after decryption of the received inbound data. In the example described herein, the inserted instructions may comprise Java instructions that call functions or APIs provided by the device operating system to write to the file system of the mobile device102. The logged data may include incoming HTTP strings as well as metadata associated with the strings such as timestamps.

The actions306,308,310, and312may be performed by a human analyst or programmer, or may be performed in an automated fashion such as by a software utility.

An action314comprises compiling or recompiling the modified source code of the application to create a modified application program316. An action318comprises installing the modified application program316on the device102. An action320comprises running or executing the modified application program316, which causes the modified application program316to write unencrypted inbound and outbound data to the log file116.

An action322, which may be performed by a human analyst or using an automated process, comprises analyzing the logged inbound and outbound data from the log file116to evaluate QoE of the application program108.

FIG. 4illustrates an example method400for evaluating QoE provided by an application program108that runs on a mobile device102and that communicates with a server106using an encrypted communication protocol. An action402comprises obtaining a network communication protocol stack110, which may be part of a device operating system. For example, the network communication protocol stack110may comprise a TCP/IP stack configured to communicate HTTP and/or HTTPS communications between the mobile device102and the application program108. The network communication protocol stack110may have an encryption/decryption layer that performs TLS or SSL encryption and decryption.

An action404comprises modifying the network communication protocol stack110to create a modified network communication protocol stack408. The modifying404may comprise configuring the network communication protocol stack408to log outbound data before encrypting the outbound data and to log inbound data after decrypting the inbound data. The inbound and outbound data may be logged to a file on the mobile device, and/or may be communicated to a server or other computer for analysis.

An action410comprises installing the modified network communication protocol stack408on the mobile device. An action412comprises running an application program108on the mobile device, wherein the application program108utilizes the modified network communication protocol stack408to send and receive information. During execution of the application program108, the modified network communication protocol stack408logs unencrypted inbound and outbound data to the log file.

An action414, which may be performed by a human analyst or using an automated process, comprises analyzing the logged data from the log file to evaluate QoE of the application program108.

FIG. 5illustrates an example method500that may be performed by the modified software component, such as a modified application or a modified network communication protocol stack, either of which may be configured to run on a mobile device or other device that communicates with and utilizes services of a network-based server.

An action502comprises receiving outbound application data. For example, a component within an application program may receive unencrypted outbound application data. As another example, a network communication protocol stack may receive outbound application data from the application program.

An action504comprises logging the received outbound application data before encryption of the received outbound application data. The outbound application data may be logged to a non-volatile file on the mobile device, as an example.

An action506comprises encrypting the outbound application data. The application data may be encrypted using any of various types of encryption, including public-key encryption. In some cases, the encryption may be performed by an encryption/decryption layer of a TCP/IP protocol stack. In other cases, the application program may be configured to perform encryption.

An action508comprises transmitting the encrypted outbound application data to a server. The application data may be transmitted over various types of network media, including either or both of wired and wireless transmission media.

An action510comprises receiving encrypted inbound application data from the server. For example, a component within the application program may receive encrypted inbound application data. As another example, the network communication protocol stack may receive inbound application from the server.

An action512comprises decrypting the inbound application data. In some cases, the decryption may be performed by an encryption/decryption layer of an TCP/IP protocol stack. In other cases, the application program may be configured to perform decryption.

An action514comprises logging the decrypted inbound application data. The inbound application data may be logged to a non-volatile file on the mobile device, as an example.

FIG. 6illustrates high-level components and functionality of an example mobile device102such as may be used to implement certain of the techniques described above. In particular, the modified software component described above, such as a modified application program or modified protocol stack, may be installed on the mobile device102in order to provide an unencrypted log of communications between the mobile device102and the server106.

In this example, the mobile device102comprises one or more processors602and associated memory604. In some embodiments, the one or more processors602may comprise one or more central processing units (CPUs), microprocessors, processing cores, graphics processing units (GPU)s, and/or other processing units or components known in the art.

The memory604may be used to store programs, program modules, data, and so forth. Programs and program modules may be executable by the one or more processors602to implement algorithms such as those described above for logging communication data. The data may include files, databases, registries, log files, etc.

In various embodiments, the memory604is volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. The memory604may include non-removable system memory as well as removable memory such as may be used to store data and to transfer and distribute software. The programs and program modules stored in the memory604may comprise methods, threads, processes, applications or any other sort of computer-executable instructions.

More specifically, the memory604may include various types of computer-readable storage media, such as electronic memory, magnetic disks, optical disks, tape, etc. Tangible computer-readable media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the mobile device102. Any such tangible computer-readable media may be part of the mobile device102or may part of a separate device that is accessible to the mobile device102.

Most relevant to the discussion herein, the memory604contains an operating system606, an application program608and a log file610. The operating system606may be responsible for basic and integral device functionality, and may also provide services for use by the application program608. For example, the operating system606may provide communication drivers612that can be accessed by the application program608for network communications. In the embodiments described herein, the communication drivers may include a TCP/IP network communication protocol stack as described above. Alternatively, or in addition, the operating system606may implement other types of communication drivers612, which may or may not include encryption/decryption services. In some cases, the communication drivers612may comprise the modified network communication protocol stack described above.

The application program608may comprise an application that runs under the supervision of the operating system606. The application program may be installed upon manufacture of the device102or may be installed by a user after purchase of the mobile device. In some cases, the application program608may comprise the modified application program described above.

The log file610may comprise a data file in the file system of the mobile device102. For example, the operating system606may implement such a file system, and the application program608may access utilities provided by the operating system606to create and write to the log file610.

The memory604may contain other programs, program modules, and/or data614that are responsible for both general and specialized operations of the mobile device102.

The mobile device102also has communication hardware616that provides physical-layer communications with other computing devices such as the server106. In some embodiments, the communication hardware616may comprise wireless transceivers, radio antennas, modems, wired connection components, etc. In embodiments in which the mobile device102comprises a smartphone, the communication hardware may include a radio-frequency (RF) radio configured to communicate using cellular telecommunication networks.