Patent Description:
Website visitor tracking is commonly used to analyse visitor behaviour and acquire sensitive data related to private individuals. Excessive tracking wastes resources, time and bandwidth, third-party trackers also follow individuals around the web and the data acquired by the trackers can be used for malicious or even criminal purposes. However, personalization of user experience using targeted advertising or content selection for example is also a result of user behaviour tracking and it cannot be fully discarded in today's interconnected world. Further background art is known from the documents <CIT>, <CIT> and <CIT>.

According to an aspect of the invention there is provided a method as specified in claims <NUM> to <NUM>.

According to other aspect of the invention, there is provided an apparatus as specified in claims <NUM> and <NUM>.

According to other aspect of the invention, there is provided a non-transitory computer-readable medium as specified in claim <NUM>.

The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

The figures and the following description relate to the example embodiments by way of illustration only. Alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

The embodiments set forth below represent the information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments.

Any flowcharts discussed herein are necessarily discussed in some sequence for purposes of illustration, but unless otherwise explicitly indicated, the embodiments are not limited to any particular sequence of steps. The use herein of ordinals in conjunction with an element is solely for distinguishing what might otherwise be similar or identical labels, such as "first message" and "second message," and does not imply a priority, a type, an importance, or other attribute, unless otherwise stated herein. The term "about" used herein in conjunction with a numeric value means any value that is within a range of ten percent greater than or ten percent less than the numeric value.

As used herein and in the claims, the articles "a" and "an" in reference to an element refers to "one or more" of the element unless otherwise explicitly specified. The word "or" as used herein and in the claims is inclusive unless contextually impossible. As an example, the recitation of A or B means A, or B, or both A and B.

Website visitor tracking is commonly used to analyse visitor behaviour and acquire sensitive data related to private individuals. Excessive tracking wastes resources, time and bandwidth, third-party trackers also follow individuals around the web and the data acquired by the trackers can be used for malicious or even criminal purposes. However, personalization of user experience using targeted advertising or content selection for example is also a result of user behaviour tracking and it cannot be fully discarded in today's interconnected world.

There are different methods to ensure user privacy and protection from tracking such as blocking access to specific domains (network-based solutions) and blocking specific requests (browser-based solutions). These methods require client-side code and often result in degraded user experience including inaccessible resources and broken websites.

When a user visits a website, several requests are made by the web browser. First one is to visit the website (main request) and this is followed by sub-requests to resources specified by the content received in response to the main request. These sub-requests can be resources located on the same domain (first-party sub-request) or external domains (third-party sub-requests). The third-party sub-requests to the same resource that are made by the same browser during visiting different websites can be used for user tracking by the third party controlling the resource, thus violating user privacy. Today, when visiting a website there may be numerous trackers loaded without user being aware.

There are needs for accurate methods to distinguish the main and sub-requests at a network layer and bringing the level of control previously available only at application layer in browser-based solutions to the network layer.

<FIG> illustrates schematically an example of a system environment for a network apparatus <NUM>. The system environment illustrated in <FIG> includes a local network <NUM> that may include one or more devices <NUM> with a client application <NUM>, the network apparatus <NUM>, a local router/switch <NUM>, and an analysis engine <NUM>. The example system also includes a service cloud <NUM>, such as a network operator's cloud and the Internet <NUM>. The analysis engine <NUM> may reside in the local network, in the service cloud <NUM> or elsewhere in the network. There may also be more than one analysis engines <NUM> thus enabling at least part of the analysis being processed in more than one analysis engines. Alternative embodiments may include more, fewer, or different components from those illustrated in <FIG>, and the functionality of each component may be divided between the components differently from the description below. Additionally, each component may perform their respective functionalities in response to a request from a human, or automatically without human intervention.

In an embodiment, the device <NUM> may communicate (A) via the network apparatus <NUM> residing in the local network <NUM>. In another embodiment, the device <NUM> may communicate (B) directly via a network gateway or a modem <NUM>, for example when the device is not in the local network <NUM>. In an embodiment, the network operators may deploy a service platform on their broadband gateways <NUM> provided to customers and in their own cloud environments <NUM>. The user device(s) <NUM> may also be configured to use the services provided by the service cloud <NUM> by one or more applications <NUM> installed on the device(s) <NUM>.

The device <NUM> may be any computer device having Internet browsing capabilities, such a smartphone, laptop or a tablet. The network apparatus <NUM> collects information e.g. about the local network <NUM>, including data about the network traffic through the local network <NUM> and data identifying devices in the local network <NUM>, such as any smart appliances and user devices <NUM>. The network apparatus <NUM> is configured to receive traffic control instructions from the analysis engine <NUM> and to process network traffic based on the traffic control instructions. Processing the network traffic through the local network <NUM>, for example, can include restricting where network traffic can travel, blocking network traffic from entering the local network <NUM>, redirecting a copy of network traffic packet or features of those packets to the analysis engine <NUM> for analysis (e.g., for malicious behaviour), or quarantining the network traffic to be reviewed by a user (e.g., via the user device <NUM>) or network administrator. In some embodiments, the functionality of the network apparatus <NUM> is performed by a device that is a part of the local network <NUM>, while in other embodiments, the functionality of the network apparatus <NUM> is performed by a device outside of the local network <NUM>.

The network apparatus <NUM> may be configured to monitor traffic that travels through the local network <NUM>. In some embodiments, the network apparatus <NUM> can be a device that is a part of the local network <NUM>. The network apparatus <NUM> can be connected to the local network <NUM> using a wired connection (e.g. via en Ethernet cable connected to a router) or using a wireless connection (e.g. via a Wi-Fi connection). In some embodiments, the network apparatus <NUM> can comprise multiple devices. In some embodiments, the network apparatus <NUM> can also perform the functions of the local network router <NUM> for the local network <NUM>.

In some embodiments, the network apparatus <NUM> may intercept traffic in the local network <NUM> by signalling to the user device <NUM> that the network apparatus <NUM> is router <NUM>. In some embodiments, the network apparatus <NUM> replaces the default gateway or gateway address of the local network <NUM> with its own internet address. In some embodiments, the local network <NUM> can be structured such that all network traffic passes through the network apparatus <NUM>, allowing the network apparatus <NUM> to physically intercept the network traffic. For example, the network apparatus <NUM> can serve as a bridge through which all network traffic must travel to reach the router <NUM> of the local network <NUM>.

The analysis engine <NUM> may receive and analyze network traffic data (e.g., forwarded by the network apparatus <NUM>) associated with devices on the computer network. The analysis engine <NUM> may be implemented within a remote system (e.g., a cloud server) or within the local network <NUM>. The analysis engine <NUM> may perform operations that are computationally expensive for the network apparatus <NUM> to perform. In some embodiments, the analysis engine <NUM> replaces the network apparatus <NUM> by performing the functionalities of the network apparatus <NUM>. In these embodiments, the local network router <NUM> may be configured to forward network traffic to the analysis engine <NUM>. In some embodiments, the analysis engine <NUM> communicates with other devices on the computer network. In some embodiments, the analysis engine <NUM> is integrated into the network apparatus <NUM>.

The local network <NUM> is a local area network (LAN) that comprises the one or more devices <NUM>, network apparatus <NUM>, and local network router <NUM>. The local network <NUM> may be used for a number of purposes, including a home network or a network used by a business. The local network <NUM> is connected to the internet <NUM>, allowing devices within the local network <NUM>, including the user device <NUM>, to communicate with devices outside of the local network <NUM>. The local network <NUM> may be a private network that may require devices to present credentials to join the network, or it may be a public network allowing any device to join. In some embodiments, other devices, like personal computers, smartphones, or tablets, may join local network <NUM>.

The internet <NUM> and the local network <NUM> may comprise any combination of LANs and wide area networks (WANs), using both wired and wireless communication systems. In some embodiments, the internet <NUM> and the local network <NUM> use standard communications technologies and protocols. Data exchanged over the internet <NUM> and the local network <NUM> may be represented using any suitable format, such as hypertext markup language (HTML) or extensible markup language (XML). In some embodiments, all or some of the communication links of the internet <NUM> and the local network <NUM> may be encrypted using any suitable technique or techniques.

The user device <NUM> is a computing device capable of receiving user input as well as transmitting and/or receiving data via the Internet <NUM> or local network <NUM>. In some embodiments, a user device <NUM> is a conventional computer system, such as a desktop or a laptop computer. Alternatively, a user device <NUM> may be a device having computer functionality, such as a personal digital assistant (PDA), a mobile telephone, a smartphone, or another suitable device. The user device <NUM> is a network device configured to communicate with the Internet <NUM> or local network <NUM>. In some embodiments, the user device <NUM> executes an application (e.g., application <NUM>) allowing a user of the user device <NUM> to interact with other network devices, such as the smart appliances, the network apparatus <NUM>, the router <NUM>, or the analysis engine <NUM>. For example, the user device <NUM> executes a browser application to enable interaction between the user device <NUM> and the network apparatus <NUM> via the local network <NUM>.

The client application <NUM> is a computer program or software application configured to run on the user device <NUM>. For example, the application <NUM> is a web browser, a mobile game, an email client, or a mapping program. The user device <NUM> can have any number of applications <NUM> installed. The application <NUM> may communicate, via the user device <NUM>, with devices inside and outside of the local network <NUM>.

The application <NUM> operates in either an active or a passive state. In a passive state, the application is running in the background of the user device <NUM> or is running on an idle device or in an idle state and may have reduced functionality. In an active state, the application <NUM> may be receiving input from a user and may have increased functionality compared to the passive state. Thus, in an active state, the application <NUM> may transmit and receive increased levels of network traffic compared to the passive state.

When observing network traffic that is passing through a network apparatus <NUM>, such as a router, connection initialization requests are captured. Relations between them are unknown. It is unknown if the requests were made by same browser session, or some other application, but following information can be extracted for further analysis:.

In HTTP cases, using referrer tag, it is possible to detect third-party request of resources. However, knowing the increasing number of HTTPS usage over HTTP, such detection becomes less relevant.

Empirical evaluation shows the following traffic patterns in web browsing activities:.

Router level communication monitoring covers all communication made by the initial household and it is not limited to browsing activities of an isolated device. In this context it is called noise. Sources of noise can be: communications initiated by operating system services, communications by user installed software and applications on household devices, multiple browsers or browser tabs communications, concurrent web pages accesses, OCSP (Online Certificate Status Protocol) queries or other. Decreasing the amount of noise has an impact on the end result and is one of the challenges.

<FIG> is a flow diagram illustrating an embodiment of a method at a network apparatus connecting one or more computer devices to a computer network.

In S201, plurality of connection requests relating to monitored network traffic passing through the network apparatus are detected.

In S202, data comprising at least a host name of a host and a time of a respective connection request is extracted for each connection request based on analyzing the plurality of connection requests.

In S204, the data is analyzed to determine whether the host is in an active state, whether the host matches a domain referrer, and an amount of time from a last connection request.

In S206, in response to detecting that the host is not in the active state, the host is not matching the domain referrer, and the amount of time from the last connection request exceeds a predetermined new session threshold, a connection request is classified as a main request.

In S208, in response to detecting that the amount of time from the last connection request is below (not exceeding) a predetermined continuous session threshold, any connection requests following the main request are classified as sub-requests.

In S210, in response to detecting, for a sub-request, that a domain of a host of the sub-request in the active state does not match a current host, the sub-request is classified as a third-party request.

In S212, further action can be taken to protect the one or more computer devices from a security threat caused by any sub-requests classified as third-party requests.

In an embodiment, the main request is a request to visit a hosted website, the sub-request is a request for resources specified by content received in response to the main request, and the third-party request is a request for resources located on an external domain.

In an embodiment, the method further comprises determining initiation of the new connection request based on detecting a source port number increase for a connection request.

In an embodiment, the method further comprises dividing the sub-requests, that is the connection requests for which the amount of time from the related last requests (last request referring to the immediately preceding request of the connection request) are not exceeding the predetermined continuous session threshold into first-party requests and third-party requests, wherein the first-party request is a request for resources located on the same domain.

In an embodiment, the method further comprises setting state of the host as not being in an active state for any connection requests for which the amount of time from the last connection request exceeds the predetermined continuous session threshold, wherein in an active state main request is completed and sub-requests are ongoing.

In an embodiment, the method further comprises monitoring a state variable for each device including a last port number and a time a last request was stored.

In an embodiment, the method further comprises using one or more machine learning models for time-state domain correlation.

In an embodiment, the method further comprises decreasing amount of noise from the monitored network traffic, wherein the sources of noise comprises one or more of the following: communications initiated by operating system services, communication by user installed software and applications on household devices, multiple browsers or browser tabs communications, concurrent web page accesses, and Online Certificate Status Protocol (OCSP) queries.

In an embodiment, the extracted data further comprises a communication timestamp, communication protocol, a Media Access Control (MAC) address of a device, a source port, a complete domain name for the host, a server name indication, a Transmission Port Protocol (TCP) window size, a total length of a packet, a referrer.

In an embodiment, taking further action to protect the one or more computer devices comprises one or more of: blocking or preventing the third-party request, blocking the third-party request if the host is blacklisted, disallowing the third-party request based on determining that the main request and the sub-request does not belong to a same company.

In an embodiment, if a host is blacklisted, then it is blocked. Otherwise it may be allowed. Thus, hosts that are not in a blacklist may be allowed as also any request classified as main requests. In an embodiment, it is determined whether the main request and the sub-requests belong to a same company/organization and if so, they can be allowed while other requests are blocked.

In an embodiment, the time between connection requests is monitored. When time between the connection requests is short, for example below a predetermined threshold, then a web session is determined to be ongoing. One of the first in each session is the main request. It can also be determined that the time before a session start (i.e. between different sessions) is longer than the time between requests in a same session.

Based on study and observations on an OS networking stack, initial algorithm can be developed. First it is required to see how the usual web page is being loaded. The first request loads the main URL that is being requested by the user, service or application. Additional resources required by the initial site for its functions, advertising and others, are then loaded in a bulk.

There are several distinctive features that outline the significance and visibility of the actually visited site: time from the last request, increase in port number and hostname attributes - length of the hostname / domain. Browsable content tends to be shorter in length than automatically generated or used to provide specific functionality. It also tends to have a lesser amount of hostname parts.

For ensuring a method according to an embodiment to work and be trackable, a per-device state variable containing the last port number and time of last request is stored. In an embodiment, two constants are predefined: a new session threshold and a continuous session threshold. These constants are used in the evaluation whether the host is in active state, that is, the main request is completed, and sub-requests are ongoing.

Stream of requests are analyzed and above-mentioned information is extracted. Whenever port is increased for a request, it is determined that a new connection is initiated. Each new request can be treated as the main request when the host is not in an active state, does not match domain referrer, is a new connection and amount of time from the last request exceeds the predetermined new session threshold. When these conditions are satisfied, the main site is determined and stored in state variable.

Following requests not exceeding the predetermined continuous session threshold are divided into first party requests (with host in state and domain matching host) and third- party requests (where domain of host in state does not match current host). On exceeding the predetermined continuous session threshold, state is set to not active.

In an embodiment, the method can be further fine-tuned by using machine learning models for time-state-domain correlation. An additional dataset collection may be created on this basis. In another embodiment, other available features may also be used in determining domains, such as TCP window size, packet size, IP options.

Turning now to <FIG> that is showing an example of a network apparatus such as a router, a switch, a <NUM> modem, or other network level apparatus.

A processor <NUM> is provided that is configured to detect connection requests relating to monitored network traffic passing through the network apparatus. Further, the processor <NUM> is configured to extract data comprising at least a host name and time of the connection request based on analyzing the detected connection requests. The processor <NUM> is further configured to analyze the extracted data to determine whether the host is in an active state, whether the host matches a domain referrer and the amount of time from the last connection request. However, in some embodiments, this analysis can also be implemented in some other device external to the apparatus <NUM>. In response to detecting that the host is not in an active state, the host is not matching the domain referrer and the amount of time from the last connection request exceeds a predetermined new session threshold, the processor <NUM> is configured to classify a connection request as a main request. In response to detecting that the amount of time from the last connection request is not exceeding/is below a predetermined continuous session threshold, the processor <NUM> is configured to classify any connection requests following the main request as sub-requests. In response to detecting that the domain of host in an active state does not match current host for a sub-request, the processor <NUM> is configured to classify the sub-request as a third-party request and to take further action to protect the one or more computer devices from a security threat caused by any sub-requests classified as third-party requests.

In an embodiment, the processor <NUM> is further configured to store data such as data related to the connection requests, state information and domain data to the database <NUM>. The database <NUM> is shown in this example as being located at the apparatus <NUM>, but it will be appreciated that the apparatus <NUM> may alternatively access a remote database. The database <NUM> may comprise necessary data collected from user devices.

The apparatus <NUM> is provided with a receiver <NUM> that receives the connection requests and responses. A transmitter <NUM> is also provided for communication with the user device and/or the outside server.

In the above description, the apparatus <NUM> is described as having different transmitter and receiver. It will be appreciated that these may be disposed in any suitable manner, for example in a single transmitter and receiver, a transceiver and so on. Similarly, a single processor <NUM> is described but it will be appreciated that the function of the processor may be performed by a single physical processor or by more than one processors.

The apparatus <NUM> is also provided with a non-transitory computer readable medium in the form of a memory <NUM>. The memory may be used to store a computer programme <NUM> which, when executed by the processor <NUM>, causes the processor <NUM> to perform the functions described above. The computer program <NUM> may be provided from an external source. In an embodiment, at least some or even all of the functions of the method can be implemented in any apparatus, for example the user device or a server.

Let us turn to <FIG> flow diagram illustrating a process for distinguishing network connection requests, according to one embodiment.

Detection of the main request and start of session is presented by following steps <NUM>,<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> of the process. User types for example mit. edu to visit the website. The state is empty, domain is not stored in state either and it is treated as a new connection initiated by a user, meaning that host is saved to state. In <NUM>, input line is read and parsed. From <NUM> where it is detected that there is no active state, <NUM> is entered to determine whether host is in state. In response to detecting that host is not in state, <NUM> is entered where it is determined whether domain of host matches the referrer. This is not the case and thus, <NUM> is entered and after determining that this is a new connection, <NUM> is entered where it is determined whether time between requests exceeds a new session threshold. If this is true, then <NUM> is entered where the host is saved to state. If in <NUM>, domain of host is detected to match the referrer, then <NUM>, <NUM> and <NUM> are entered where host is saved to state, it is switched to active state and main request is current host.

Redirection is presented in process steps <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. If web content is stored under web. edu, user is redirected from mit. edu to web. Host is in state and request domain matches current hosts. According to timing (time delta is less than a continuous session threshold time), it is switched to active state (meaning that this resource is currently being actively accessed by user) and since domain of host matches current host domain (mit. edu) it qualifies as being a first-party sub-request.

Process steps <NUM>, <NUM>, <NUM>, <NUM> show an example of how the first-party requests (requests from mit. edu) are treated by evaluating whether the new host qualifies as new session or not. Process steps <NUM>, <NUM>, <NUM>, <NUM>, <NUM> outline the third-party detection where domains are not matching the current host's and the request still does not qualify for a new session. This means that this is a third-party request and can be treated as a possible tracker.

Steps <NUM>, <NUM> and <NUM> show end of process when in <NUM> it is detected that time delta exceeds a new session threshold. When it is detected that the domain does not match current host in <NUM>, then in <NUM> it is switched to inactive state and the host is removed from state in <NUM>.

For a website to function correctly, resources from the same domain or set of domains must be loaded. This includes images, CSS (Cascading Style Sheet) or JS (JavaScript) resources and other. Each third-party resource can be treated as a tracker, possibly adding additional functionality of social networking, advertisement etc. User who wants to experience a clean Internet browsing and leave as little of digital footprint as possible, can choose to disallow third-party requests to track his/her activities. Thus, when the user accesses the website, all the mandatory resources are loaded while third-party sub-requests can be blocked. This enables a faster and cleaner Internet browsing experience.

In example embodiments, there are provided classification systems that can perform one or more of the following functions:.

Machine learning, rules and/or other machine learning models may here be utilized for estimating the current state of host. The nature of the model used by the system may be, or may incorporate elements, from one or more of the following: a neural network trained using a training data set, exact or heuristic rules (e.g. hardcoded logic), fuzzy logic based modelling, and statistical inference-based modelling. The model may be trained to consider particular patterns, data, processes, connections, and dependencies between processes.

It will be appreciated that various modifications may be made to the above described embodiments without departing from the scope of the present invention. For example, the database or analysis engine may be in separate entities to the apparatus, in which case the apparatus will send queries remotely to the analysis engine.

The steps, signalling messages and related functions described above in relation to the figures are in no absolute chronological order, and some of the steps may be performed simultaneously or in a different order. Other functions may also be executed between the steps and other signalling may be sent between the illustrated ones. Some of the steps can also be left out or replaced by a corresponding step. The system functions illustrate a procedure that may be implemented in one or more physical or logical entities.

The techniques described herein can be implemented by various means. An apparatus or system that implements one or more of the described functions may comprise not only existing means but also means for implementing one or more functions of a corresponding apparatus that is described with an embodiment. An apparatus or a system may also comprise separate means for each separate function. For example, the embodiments may be implemented in one or more modules of hardware or combinations thereof. For software, implementation can be through modules, for example such procedures and functions that perform the functions described. The software code may be stored in any suitable data storage medium that is readable by processors, computers, memory units or articles of manufacture, and may be executed by one or more processors or computers. The data storage medium or memory unit or database may be implemented within the processor or computer apparatus, or as an external part of the processor or computer apparatus.

The programming, such as executable code or instructions, electronic data, databases or other digital information may be stored into memories and can include a processor-usable medium embodied in any computer program product which can contain, store, or maintain programming, data or digital information for use by or in connection with an instruction execution system, such as the processor.

An embodiment provides a non-transitory computer-readable medium comprising stored program code comprised of computer-executable instructions. The computer program code comprises a code for detecting connection requests relating to monitored network traffic passing through the network apparatus, a code for extracting data comprising at least a host name and time of the connection request based on analyzing the detected connection requests, a code for analyzing the extracted data to determine whether the host is in an active state, whether the host matches a domain referrer and the amount of time from the last connection request, a code for classifying a connection request as a main request in response to detecting that the host is not in an active state, the host is not matching the domain referrer and the amount of time from the last connection request exceeds a predetermined new session threshold, a code for classifying any connection requests following the main request as sub-requests in response to detecting that the amount of time from the last connection request is below a predetermined continuous session threshold, a code for classifying the sub-request as a third-party request in response to detecting that the domain of host in an active state does not match current host for a sub-request. In an embodiment, it further comprises a code for taking further action to protect the one or more computer devices from a security threat caused by any sub-requests classified as third-party requests.

Claim 1:
A method comprising:
detecting (<NUM>) a plurality of connection requests relating to monitored network traffic passing through a network apparatus;
extracting (<NUM>) data comprising at least a host name of a host, which is configured to host resources, and a time of a respective connection request for each connection request of the plurality of connection requests;
analyzing (<NUM>), for any connection request, the data to determine whether the host is in an active state, the active state meaning that resources hosted by the host are currently being actively accessed, whether the host matches a domain referrer of a Hypertext Transfer Protocol, HTTP, request header of the connection request, and an amount of time from an immediately preceding last connection request;
in response to detecting that the host is not in the active state, the host does not match the domain referrer, and the amount of time from the immediately preceding last connection request exceeds a predetermined new session threshold, classifying (<NUM>) a connection request as a main request and switching the host to the active state;
in response to detecting that the amount of time from the immediately preceding last connection request is below a predetermined continuous session threshold, classifying (<NUM>) any connection requests following the main request as sub-requests;
in response to detecting, for a sub-request, that a domain of the host in the active state does not match a domain of a current host for the sub-request, classifying (<NUM>) the sub-request as a third-party sub-request, or else classifying the sub-request as a first-party sub-request; and
taking (<NUM>) further action to protect one or more computer devices of a computer network system from a security threat caused by the third-party sub-request.