Patent Description:
Digital resources can be accessed by large numbers of remotely located users. Determining the number of users that access digital resources can be performed, e.g., as part of managing the distribution and storage of digital resources.

The extent to which a process is privacy preserving can be measured in various ways. For example, differential privacy techniques can be used to quantify the extent to which processes are privacy preserving. A computational process operating on a dataset can be referred to as being privacy sensitive (or privacy preserving), e.g., if the process is adapted to prevent the leakage of information from the dataset. <CIT> describes an online excavation method of an active user website access mode.

This specification describes a system implemented as computer programs on one or more computers in one or more locations that can perform efficient and privacy sensitive estimation of digital resource access frequency.

According to a first aspect there is provided a method performed by one or more computers, the method comprising: obtaining access data for a digital resource, wherein the access data comprises, for each time point in a sequence of time points, data identifying a set of users that accessed the digital resource at the time point; generating a tree model based on the access data, wherein each node of the tree model is associated with a respective access value that characterizes a number of users that satisfy a node-specific selection criterion based on accessing the digital resource; selecting, for each node in the tree model, a respective private access value for the node that: (i) is an approximation of the access value for the node, and (ii) is selected from a finite set of possible private access values; receiving a request to determine a number of users that accessed the digital resource at least a predefined number of times within a time window; and in response to the request: generating the estimate for the number of users that accessed the digital resource at least the predefined number of times within the time window based on private access values associated with one or more nodes in the tree model.

In some implementations, selecting, for each node in the tree model, a respective private access value for the node comprises, at each of one or more iterations starting from a first iteration in a sequence of iterations: receiving (i) a current set of one or more sub-trees of the tree model and (ii) a current threshold value; classifying, for each node in each sub-tree in the current set of sub-trees, whether the access value for the node satisfies an acceptance criterion based on the current threshold value; identifying one or more nodes in the tree model as being target nodes for the iteration based on the classification of the nodes in the sub-trees in the current set of sub-trees; and selecting a private access value for each of the target nodes based on the current threshold value.

In some implementations, at the first iteration in the sequence of iterations, the current set of sub-trees comprises the tree model.

In some implementations, the method further comprises: identifying a next set of one or more sub-trees of the tree model, based on the classifications of the nodes in the sub-trees in the current set of sub-trees; and providing the next set of sub-trees for processing at a next iteration in the sequence of iterations.

In some implementations, for each iteration after a first iteration in the sequence of iterations, the current threshold value for the next iteration is less than the current threshold value for a preceding iteration in the sequence of iterations.

In some implementations, identifying one or more nodes in the tree model as being target nodes for the iteration based on the classifications of the nodes in the sub-tree in the current set of sub-trees comprises: identifying one or more transition nodes, wherein each transition node: (i) is included in a sub-tree in the current set of sub-trees, (ii) is classified as satisfying the acceptance criterion, and (iii) does not have child nodes that are classified as satisfying the acceptance criterion; and designating each transition node as being a target node.

In some implementations, the method further comprises designating each node that: (i) is included in a sub-tree in the current set of sub-trees, and (ii) is an ancestor node to a transition node, as being a target node.

In some implementations, at each of one or more iterations in the sequence of iterations, selecting a private access value for each of the target nodes based on the current threshold value comprises: selecting a same private access value for each of the target nodes identified at the iteration.

In some implementations, the selected private access value is within a tolerance range around the current threshold value.

In some implementations, identifying a next set of one or more sub-trees of the tree model based on the classifications of the nodes in the sub-trees in the current set of sub-trees comprises, for each transition node: designating one or more disjoint sub-trees of the transition node as being included in the next set of sub-trees.

In some implementations, classifying, for each node in each sub-tree in the current set of sub-trees, whether the access value for the node satisfies an acceptance criterion based on the current threshold value comprises, for one or more levels in one or more sub-trees in the current set of sub-trees: determining that the level in the sub-tree includes at least one node having an access value that exceeds the current threshold value; and for each node at the level in the sub-tree: generating a transformed access value for the node, comprising combining noise with the access value for the node; and classifying the access value for the node as satisfying the acceptance criterion if the transformed access value for the node exceeds the current threshold value.

In some implementations, generating the transformed access value for the node further comprises adding one or more offsets to the access value for the node.

In some implementations, combining noise with the access value for the node comprises: sampling a noise value from a probability distribution; and adding the sampled noise value to the access value for the node.

In some implementations, the probability distribution is a truncated Laplace distribution.

In some implementations, determining that the level in the sub-tree includes at least one node having an access value that exceeds the current threshold value comprises: determining that the level in the sub-tree includes at least one node having an access value that exceeds the current threshold value using a sparse vector technique.

In some implementations, for each node of the tree model: the node is associated with a respective key that specifies one or more time intervals; and the node-specific selection criterion for the node based on the one or more time intervals specified by the key for the node.

In some implementations, generating the estimate for the number of users that accessed the digital resource at least the predefined number of times within the time window based on private access values associated with one or more nodes in the tree model comprises: identifying a plurality of nodes in the tree model that each have a respective key which satisfies a selection criterion based on the time window; determining a combination of the private access values associated with the identified nodes; and generating the estimate for the number of users based at least in part on the combination of the private access values associated with the identified nodes.

In some implementations, determining the combination of the private access values associated with the identified nodes comprises: determining a sum of the private access values associated with the identified nodes.

In some implementations, identifying the plurality of nodes in the tree model comprises: identifying that one or more leaf nodes of the tree model and one or more internal nodes of the tree model each have a respective key with satisfies the selection criterion based on the time window.

In some implementations, a user accesses the digital resource by receiving a transmission of a digital component.

In some implementations, the tree model is a binary tree model.

In some implementations, the method further comprises: outputting the estimate for the number of users that accessed the digital resource at least the predetermined number of times within the time window.

In some implementations, the method further comprises modifying the digital resource based on the estimate for the number of users that accessed the digital resource.

In some implementations, modifying the digital resource comprises modifying a bit rate or resolution of the digital resource.

In some implementations, the method further comprises modifying the conditions under which the digital resource is made available to users based on the estimate for the number of users that accessed the digital resource.

In some implementations, modifying the conditions under which the digital resource is made available to users comprises modifying distribution criteria or computational resource availability.

In another aspect, there is provided a system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the methods described herein.

In another aspect, there is provided one or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the methods described herein.

The system described in this specification can generate accurate and privacy preserving access frequency estimates. An access frequency estimate can refer to an estimate for a number of users that accessed a digital resource at least a threshold number of times during a time window. Access frequency estimates generated by the system can be referred to as being privacy preserving (or privacy sensitive), e.g., because they are adapted to prevent leakage of information about users. For instance, access frequency estimates generated by the system can satisfy criteria for being differentially private, e.g., such that the access frequency estimates cannot be leveraged to determine when and if individual users have accessed the digital resource.

The system can generate access frequency estimates that are accurate when accuracy is measured in terms of either additive error or multiplicative error, in contrast to some conventional techniques which are designed to minimize only additive error without reference to multiplicative error.

To enable generation of privacy preserving access frequency estimates, the system can process access data for a digital resource (e.g., that identifies a respective set of users that accessed the digital resource at each of multiple time points) to generate a tree model (data structure). A tree model can refer to a hierarchical structure that includes a set of nodes and a set of edges, where each edge connects a respective "parent" node to a respective "child" node, and where each node (except for a root node) has exactly one parent node.

The system can construct the tree model to encode the access data such that an access frequency estimate to be generated for any time window by combining access values associated with nodes in the tree model. To preserve privacy, the system can generate a respective "private" access value for each node in the tree model and generate access frequency estimates using the private access values for the nodes rather than the original access values.

The system can generate the private access values for the nodes in a manner that causes the access frequency estimates generated using the private access values to be privacy preserving. In particular, to generate the private access values, the system can classify each access value into an interval created by a pair of thresholds from a sequence of thresholds, and then map each access value onto a private access value representing the interval that includes the access value.

To this end, the system implements an iterative classification technique, where at each iteration, the system identifies nodes having access values that are above a corresponding threshold, and at the next iteration considers only nodes having access values below the threshold. The system can efficiently classify access values as being above or below a threshold by identifying "transition" nodes. A transition node refers to a node that: (i) has an access value that exceeds the threshold, and (ii) does not have child nodes with access values that exceed the threshold. Given the locations of the transition nodes, the system can immediately classify the other nodes. In particular, if a node is an ancestor of a transition node, then the access value for the node is above the threshold; otherwise, the access value for the node is below the threshold. Efficiently classifying access values relative to thresholds enables the system to: (i) reduce privacy loss, (ii) increase the accuracy of access frequency estimates generated using the private access values, and (iii) reduce consumption of computational resources, e.g., memory and computing power.

The system described in this specification enables reduced consumption of computational resources, e.g., memory and computing power, by enabling access frequency estimates to be generated for any time interval by combining values associated with a limited number of nodes in a tree model. In contrast, some conventional approaches for generating access frequency estimates require storing an entire set of access data (which requires more memory than storing the tree model) and processing the entire set of access data each time an access frequency estimate is generated (which consumes significantly more computing power than generating access frequency estimates by leveraging the tree model).

<FIG> shows an example access estimation system <NUM>. The access estimation system <NUM> is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

The system <NUM> is configured to respond to query requests <NUM> regarding access data <NUM>. Each query request <NUM> defines a request to produce an estimate for how many users accessed a digital resource at least a specified minimum number of times (e.g., <NUM> time, or <NUM> times, or <NUM> times, or <NUM> times, or any other appropriate number of times) in a time window specified by the query request. The system <NUM> responds to query requests <NUM> by producing corresponding privacy sensitive estimates <NUM> for the number of users that accessed the digital resource at least the specified minimum number of times in the time window specified by the query request.

The access data identifies, for each time point in a sequence of time points, a set of users that accessed the digital resource at the timepoint. A user can access a digital resource in any of a variety of possible ways. A few examples of possible ways that users can access digital resources are described next.

In some implementations, the digital resource can be a data storage location, e.g., a database. A user can access a database, e.g., through an application programming interface (API) made available by the database, e.g., to query information from the database, or to store information in the database, or both.

In some implementations, the digital resource can be an application or a webpage, e.g., which a user can access by way of a user interface made available on a user device. Accessing an application or a webpage can refer to interacting with the application or webpage.

In some implementations, the digital resource can be a computational resource, e.g., one or more processors (e.g., central processing units (CPUs) or graphics processing units (GPUs)) made available to users in a cloud computing environment. A user can access a computational resource, e.g., by providing data defining one or more computational tasks to be performed using the computational resource.

In some implementations, the digital resource can be a digital component, and a user can access the digital resource by receiving a transmission of the digital resource at a user device, e.g., from a digital component transmission system. An example of a digital component transmission system is described with reference to <FIG>.

As used throughout this document, the phrase "digital components" refers to discrete units of digital content or digital information that can include one or more of, e.g., video clips, audio clips, multimedia clips, images, text segments, or uniform resource locators (URLs). A digital component can be electronically stored in a physical memory device as a single file or in a collection of files, and digital components can take the form of video files, audio files, multimedia files, image files, or text files and include streaming video, streaming audio, social network posts, blog posts, and/or advertising information, such that an advertisement is a type of digital component. Generally, a digital component is defined by (or provided by) a single source (e.g., a digital component provider), but a digital component provided from one source could be enhanced with data from another source (e.g., weather information, real time event information, or other information obtained from another source).

The request <NUM> can specify information determining how the system <NUM> generates the estimate <NUM>. For example, the request <NUM> can specify the given time window for which the system <NUM> produces the estimate <NUM>. As another example, the request <NUM> can specify a minimum number and the system <NUM> can generate the estimate <NUM> for the number of users that accessed the digital resource a number of times exceeding the specified minimum. As another example, the system <NUM> can obtain access data <NUM> for multiple digital resources, and the request <NUM> can specify the particular digital resource for which the system <NUM> produces the estimate <NUM>.

The system <NUM> can use an access frequency estimate, e.g., an estimate for a number of users that accessed the digital resource at least a predefined number of times in a time window, in a variety of possible ways, a few examples of which are described next.

In some implementations, the system <NUM> can generate a notification indicating an access frequency estimate, and provide the notification, e.g., for display by way of a user interface.

In some implementations, the system <NUM> can use an access frequency estimate to determine whether a cvberattack, e.g., a distributed denial-of-service (DDoS) attack, has occurred. For instance, the system can determine that a cyberattack has occurred based at least in part on a determination that at least a threshold number of users have accessed the digital resource at least a threshold number of times within a time window.

In some implementations, the system <NUM> can modify conditions under which the digital resource is available to users based on an access frequency estimate. For instance, if users access the digital resource by receiving transmissions of a digital component, the system <NUM> can modify distribution criteria controlling when the digital resource is transmitted to users based on an access frequency estimate, e.g., to cause the digital component to be transmitted to more or fewer users.

In some implementations, the system <NUM> can modify a digital resource based on an access frequency estimate. For instance, if the digital resource is a computational resource (e.g., in a cloud computing environment), then the system <NUM> can increase or decrease the computing power available by way of the computational resource in response determining an access frequency estimate satisfies a threshold. If the digital resource is a digital component, the component could be modified such as a resolution of an image increased or decreased, or bit rate of audio or video increased or decreased dependent on an access frequency estimate to control bandwidth used during distribution of the digital resource.

The system <NUM> includes a tree generation system <NUM>, a selection system <NUM>, and an estimation system <NUM>, which are each described in more detail below.

The tree generation system <NUM> is configured to process the access data <NUM> to produce a tree model <NUM> based on the access data <NUM>. The tree model <NUM> structures information from the access data <NUM> for the purpose of generating the estimates <NUM>. The tree generation system <NUM> associates each node of the tree model <NUM> with a particular time window for the node. The tree generation system <NUM> also associates each node of the tree model <NUM> with a respective access value that specifies a number of times that users have accessed the digital resource within the particular time window assigned to the node. The system <NUM> can generate estimates <NUM> by producing privacy sensitive approximations of the access values assigned to the nodes of the tree model <NUM> and computing the estimates <NUM> based the approximate values for the nodes required to respond to the query requests <NUM>. Specific details regarding an example tree model <NUM> are described in more detail below with reference to <FIG>. An example process for generating the tree model <NUM> from the access data <NUM> is described in more detail below with reference to <FIG>.

The selection system <NUM> can process the tree model <NUM> to produce private access values <NUM> for each of the nodes within the tree model. For each node of the tree model <NUM>, the selection system <NUM> assigns one of a finite set of private access values to the node that approximates the access value assigned to the node in a manner that preserves user privacy. An example process for selecting private access values <NUM> for the tree model <NUM> is described in more detail below with reference to <FIG>.

The estimation system <NUM> can process the private access values <NUM> and the query request <NUM> to produce the estimate <NUM>. In particular, the estimation system <NUM> can determine the relevant nodes having information required to respond to the query request <NUM> and can generate the estimate <NUM> based on the private access values <NUM> assigned to the relevant nodes. An example process for processing the private access values <NUM> is described in more detail below with reference to <FIG>.

The system <NUM> can produce estimates <NUM> of how many users accessed the digital resource that are accurate (e.g., to within a threshold accuracy tolerance) while remaining privacy sensitive, e.g., by reducing the likelihood that the estimate can reveal private user information from the access data <NUM>. For example, the estimates <NUM> can satisfy differential privacy criteria such that the estimates <NUM> do not reveal information regarding whether any particular user accessed the digital resource. The criterion by which the estimates <NUM> are considered privacy sensitive and accurate are described next.

For each node, u, of the tree model <NUM>, <IMG>, (i.e., u ∈ nodes(<IMG>) ), the selection system <NUM> assigns a private access value to the node, w̃u, that approximates the access value of the node, wu. An estimate <NUM>, E(<IMG>), is referred to as (ε, δ)-differentially private if, for any neighboring tree, <IMG>, (i.e., any tree constructed by incrementing or decrementing the access value of exactly one of the nodes of the tree <IMG>), the following criterion is satisfied:
<MAT>.

Where S is any subset of the estimates <NUM> that can be returned by the system <NUM>. When appropriately configured according to this specification, the system <NUM> can produce (ε, δ)-differentially private estimates <NUM>.

The estimates <NUM> are considered (α, η)-accurate, with respect to node-specific thresholds, τu, when, for any node u, the following criterion holds: <MAT>.

<FIG> is a flow diagram of an example process for access estimation. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, a resource estimation system, e.g., the resource estimation system <NUM> of <FIG>, appropriately programmed in accordance with this specification, can perform the process <NUM>.

The system obtains access data for a digital resource (step <NUM>). The access data identifies sets of users that accessed the digital resource at time points in a sequence of time points. The sequence of time points can include any appropriate number of time points, e.g., <NUM> time points, or <NUM> time points, or <NUM>,<NUM>,<NUM> time points. The time points can be separated, e.g., by uniform values, e.g., of <NUM> second, or <NUM> minute, or <NUM> hour, or non-uniform values. A user can be said to access a digital resource "at a time point," e.g., if the user accesses the digital resource in a time window parametrized by the time point, e.g., a time window defined by the interval of time between the time point and the next time point.

The system can obtain the access data by any appropriate method. For example, the system can monitor user access to the digital resource and update the access data as users access the digital resource. As another example, the system can receive access data from an external source (e.g., a system that generates the access data by monitoring user access to the digital resource).

The system generates a tree model based on the access data (step <NUM>). The system associates each node of the tree model with a respective access value that characterizes a number of users that satisfy a node-specific selection criterion based on accessing the digital resource. An example process for generating the tree model <NUM> from the access data <NUM> is described in more detail below with reference to <FIG>.

Based on the generated tree model <NUM>, the system can determine multiple parameters to ensure that the estimates <NUM> are both (ε, δ)-differentially private and (α, η)-accurate.

The system can receive or select an appropriate minimum threshold, τmin, satisfying: <MAT>.

Where d is the depth of the tree model <NUM>. The estimates <NUM> can be (α, η)-accurate with respect to the threshold τmin.

The system can receive or select a parameter β such that: <MAT>.

The system can receive or select a parameter R, following: <MAT>.

The system can receive or determine an approximate root access value, M, for tree model <NUM> following: <MAT>.

Where wroot is the access value for the root node of the tree model <NUM> and nM is a random variable sampled from a truncated Laplace distribution having parameter <MAT> and range R. A truncated Laplace distribution having parameter σ and range R has density proportional to <MAT> as restricted to x ∈ [-R, R].

The system <NUM> can receive or determine the number of private access values within the set of private access values, l, following: <MAT>.

The system <NUM> can receive or determine the set of private access values {M<NUM>,. , Ml}, with each Mi defined following: <MAT>.

The system selects respective private access values for the each of the nodes in the tree model (step <NUM>). The private access value assigned to a given node approximates the access value for the given node and is selected from a finite set of possible private access values. An example process for selecting private access values for the tree model is described in more detail below with reference to <FIG>.

The system receives a request to determine a number of users that accessed the digital resource at least a predefined number of times within a time window (step <NUM>).

In response to the request, the system generates an estimate for the number of users that accessed the digital resource at least the predefined number of times within the time window based on private access values associated with nodes in the tree model (step <NUM>). An example process for processing the private access values to generate the estimate is described in more detail below with reference to <FIG>.

<FIG> illustrates an example tree model <NUM>. The model tree <NUM> includes multiple nodes and structures the nodes using parent-child relationships among the nodes.

The particular tree model <NUM> depicted in <FIG> is provided solely for the purpose of explaining concepts presented in this specification. The actual model trees (e.g., the model tree <NUM>) used in the operation of the systems and processes described in the specification can be significantly larger (in terms of the number of nodes in the tree, the width of the tree, and the depth of the tree) and can have different relationships between the nodes of the tree.

Each node in the tree can have one or more child nodes. For example, nodes <NUM> and <NUM> are the child nodes of node <NUM>. As a further example, the nodes <NUM> and <NUM> are the child nodes of node <NUM>.

The nodes within the tree <NUM> that do not have child nodes are referred to as leaf nodes. For example, nodes <NUM>, <NUM>, <NUM>, and <NUM> are the leaf nodes of the tree <NUM>.

When a given node has one or more child nodes, the given node is referred to as the parent node of the child nodes. Exactly one node of the tree has no parent node and is referred to as the root node of the tree. For example, node <NUM> is the root node of tree <NUM>.

Aside from the root node <NUM>, every node within the tree <NUM> has exactly one parent node. For example, the node <NUM> is the parent node for nodes <NUM> and <NUM>. As a further example, the node <NUM> is the parent node for nodes <NUM> and <NUM>.

The model tree <NUM> can have any organization suitable for structuring the access data <NUM> for the purpose of generating the estimate <NUM>. For example, each node of the model tree <NUM> can have a distinct number of children. As another example, the model tree <NUM> can be a binary, ternary, or m-ary tree, where the nodes of the model tree <NUM> have a fixed number of children. As a particular example, the model tree <NUM> is a binary tree if every node that is not a leaf node of the tree <NUM> has exactly <NUM> child nodes.

For every sequence of nodes such that each node in the sequence is the parent node of the next node of the sequence (herein called a connecting sequence), the first node of the sequence is referred to as an ancestor of the last node of the sequence and the last node of the sequence is referred to as a descendent of the first node. For example, nodes <NUM> and <NUM> are the ancestors of node <NUM>. As a further example, nodes <NUM>, <NUM>, <NUM>, and <NUM> are some (but not all) of the descendants of node <NUM>. The root node <NUM> is an ancestor of all other nodes within the tree <NUM>.

For each pair of nodes where the first node is an ancestor of the second node, there is a unique connecting sequence between the pair. In particular, there is a unique connecting sequence between the root node <NUM> of the tree <NUM> and every other node of the tree <NUM>. The length of the connecting sequence between the root node <NUM> and a given node of the tree <NUM> is referred to as the depth of the given node within the tree <NUM>. For example, the root node has a depth of one in the tree <NUM>. As another example, the nodes <NUM> and <NUM> have depths of two within the tree <NUM>. As another example, the nodes <NUM>, <NUM>, <NUM>, and <NUM> have depths of three within the tree <NUM>.

A collection of all nodes having a same depth in the tree <NUM> is referred to as being at the same level in the tree <NUM>. For example, the node <NUM> is at level <NUM> in the tree <NUM>. As another example, the nodes <NUM> and <NUM> are at level <NUM> in the tree <NUM>. As another example, the nodes <NUM>, <NUM>, <NUM>, and <NUM> are at level <NUM> in the tree <NUM>.

A sub-tree of the tree <NUM> is a collection of nodes that: (i) have the same parent-child relations between the nodes of the sub-tree as the tree <NUM> and (ii) has exactly one node, referred to as the root node of the sub-tree, that is an ancestor of all other nodes in the sub-tree. For example, the collection of nodes <NUM>, <NUM>, and <NUM> forms a sub-tree of the tree <NUM> that has node <NUM> as the root node of the sub-tree.

Nodes within a sub-tree of the tree <NUM> can be referred to using similar terminology as described above. As a particular example, the nodes <NUM> and <NUM> can be referred to as the leaf nodes of the sub-tree formed by the collection of nodes <NUM>, <NUM>, and <NUM>.

Two sub-trees of the tree <NUM> are referred to as disjoint when the two sub-trees share no nodes in common. For example, the sub-tree formed by the collection of nodes <NUM>, <NUM>, and <NUM> is disjoint to the sub-tree formed by the collection of nodes <NUM>, <NUM> and <NUM>.

For every given node within the tree <NUM>, there is a unique sub-tree, referred to as the descendant sub-tree for the given node, formed by the given node and all of its descendants within the tree <NUM>. For example, the sub-tree formed by the collection of nodes <NUM>, <NUM>, and <NUM> is the descendant sub-tree of the node <NUM>. As another example, the tree <NUM> is the descendant sub-tree of the node <NUM>. As another example, the collection containing only the node <NUM> is the descendant sub-tree of the node <NUM>.

Multiple data values can be associated with each of the nodes of the tree <NUM>. The inset <NUM> shows example data values associated with the node <NUM>. For example, an access value <NUM>-A and a private access value <NUM>-B are assigned to the node <NUM> in the tree <NUM>. As another example, a time window <NUM>-C can be associated with the node <NUM>.

Other data values can be assigned to the nodes of the tree <NUM> to assist in computing the estimate <NUM>. For example, a query key <NUM>-D can be assigned to the node <NUM> for use in determining whether the node <NUM> is associated with relevant information to the query request <NUM>. As another example, a classification <NUM>-E can be assigned to the node <NUM> as part of the process of selecting the private access value <NUM>-B for the node <NUM>.

The access value assigned to a given node specifies a number of times that users have accessed the digital resource within the time window associated with the node. For example, the access value <NUM>-A specifies the number of times that users have accessed the digital resource within the time window <NUM>-C.

The time window of a given parent node can be the union of the time windows associated with the child nodes of the given parent node and the time windows of the child nodes are referred to as being included within the time-window of the parent node. The tree <NUM> is referred to as a partition of time windows if: (i) the time windows of each particular node within the tree <NUM> are included within the time windows of every ancestor of the particular node and (ii) a particular time falls into the time window of at most one node at any given level of the tree <NUM>.

The tree <NUM> can be, but does not need to be, a binary search tree for time windows of the access data <NUM>. The tree <NUM> is a binary search tree for time windows of the access data <NUM> if: (i) the time window of the root node <NUM> includes every time a user accessed the digital resource as specified by the access data <NUM>, (ii) the tree is a binary tree, and (iii) the tree <NUM> partitions the time window of the root node <NUM>.

<FIG> is a flow diagram of an example process for generating a tree model. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, a tree generation system, e.g., the tree generation system <NUM> of <FIG>, appropriately programmed in accordance with this specification, can perform the process <NUM>.

The tree generation system receives access data (step <NUM>).

The tree generation system determines the structure for the generated tree model (step <NUM>). In particular, the tree generation system determines a number of nodes for the tree model and determines parent-child relationships for the nodes.

The tree generation system can determine the tree model structure in any of a variety of ways. For example, the tree model structure can be predetermined and the tree generation system can map information from the access data into the predetermined structure of the tree model. As a further example, the tree model may be generated with as a binary search tree with a pre-determined number of leaf nodes associated with time windows of the same length of time.

As another example, the tree model structure can be determined based on the access data. As a further example, tree model structure can follow an iterative generative process. At the first iteration of the iterative generative process, the structure can be initialized as a single root node associated with the entire time window of the access data. At each following iteration, the tree model structure can be updated by adding child nodes for every leaf node of the current structure associated with a time window in which the access data specifies that the users accessed the digital resource more than a minimum number of times. The new child nodes for a given leaf node can be associated with time windows such that the time windows of the new child nodes divide, as evenly as possible, the number of user accesses to the digital resource encompassed by the time window of the given leaf node. The iteration can continue until every leaf node of the structure is associated with a time window encompassing no more than the minimum number of user accesses to the digital resource. The tree structure obtained after the final iteration can be used as the structure for the generated tree model.

The tree generation system can associate query keys to every node in the tree model (step <NUM>). The query key for a particular node identifies whether the particular node is associated with data relevant to generating the estimation <NUM> in response to the query request <NUM>. As a particular example, the query key for a particular node can characterize the time window associated with the particular node and can identify whether the time window of the particular node falls within a time window specified by the query request <NUM>.

The tree generation system associates access values to every node in the tree model (step <NUM>). In particular, the tree generation system can assign the number of times that users have accessed the digital resource within the time window associated with a particular node as the access value for the particular node.

The tree generation system finally returns the tree model (step <NUM>).

<FIG> is a flow diagram of an example process for selecting private access values. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, a selection system, e.g., the selection system <NUM> of <FIG>, appropriately programmed in accordance with this specification, can perform the process <NUM>.

The selection system can iteratively perform the process <NUM> over a sequence of iterations. In particular, each iteration of the process <NUM> can correspond to one of the finite set of private access values. For example, if the set of private access values is the set {Ml,. , M<NUM>}, the selection system can perform l iterations, with the j-th iteration corresponding to the private access value Ml+<NUM>-j.

At each iteration, the selection system can determine a number of iteration specific parameters.

During the iteration corresponding to the private access value Mi, the selection system can receive or determine an iteration specific threshold, τi, following: <MAT>.

The selection system can receive or determine a parameter, C, following: <MAT>.

During the iteration corresponding to the private access value Mi, the selection system can receive or determine iteration specific noise and error parameters, ηi, εi, and δi, following: <MAT> <MAT> <MAT>.

During the iteration corresponding to the private access value Mi, the selection system can receive or determine an iteration specific cutoff, ci, and range, Ri, following: <MAT> <MAT>.

For each iteration, the selection system can receive a current set of sub-trees of the tree model and a current threshold for the iteration (step <NUM>). In some implementations, at the first iteration, the selection system can receive the model tree as the set of sub-trees for the first iteration. In some implementations, at each iteration after the first iteration, the current threshold for the iteration is strictly less than the threshold used for the previous iteration.

For each current sub-tree of the iteration, the selection system can classify each node of the sub-tree based on the current threshold for the iteration (<NUM>). The selection system classifies a given node by assigning an appropriate classification to the given node. The selection system classifies a given node using an acceptance criterion based on the current threshold. An example process for classifying the access values of nodes is described in more detail below with reference to <FIG>.

The selection system can then identify one or more nodes of the tree model as being target nodes for the current iteration based on the classifications the system has assigned to the nodes. The target nodes for the current iteration are the nodes to which the selection system will assign private access values at the end of the current iteration.

In some implementations, the selection system can determine the current target nodes by first identifying one or more transition nodes within the current set of sub-trees for the iteration (step <NUM>). The selection system identifies a particular node in the current set of sub-trees as being a transition node when: (i) the particular node is classified as satisfying the acceptance criterion for the current iteration and (ii) every child note of the particular node is classified as not satisfying the acceptance criterion.

The selection system can then identify nodes within the current set of sub-trees as being target nodes based on the classification of the nodes for the current iteration (step <NUM>). In implementations where the system has identified nodes as transition nodes for the current iteration, the system identifies the current transition nodes as being target nodes for the current iteration. In some implementations where the system has identified nodes as transition nodes for the current iteration, the system can identify each ancestor of each current transition node as being a target node for the current iteration.

The selection system can then select private access values to associate with the target nodes for the current iteration based on the current threshold (step <NUM>). The selection system assigns a private access value to a given node by selecting one private access value for the given node from a finite set of private access values. In some implementations, selects a same private access value for the current iteration and assigns the same private access value to every target node for the current iteration.

In some implementations where the same private access value is assigned to every target node for the current iteration, the selection system can select the same private access value within a certain tolerance of the current threshold. For example, the selection system can assign the private access value Ml+<NUM>-i to all of the target nodes for the i-th iteration.

The selection system then assigns the private access values for the target nodes of the current iteration (step <NUM>). The selection system can assign the private access values to the corresponding nodes of the tree model.

In some implementations, the selection system can identify the set of sub-trees to be used in the next iteration. In some implementations where the selection system identifies the set of sub-trees for the next iteration, system can select a set of one or more disjoint sub-trees of the tree model where the root node of each sub-tree is a descendant of one of the transition nodes for the current iteration.

The selection system can then determine whether the selection of private access values is complete (step <NUM>). If the selection of private access values is complete (e.g., after l iterations), then selection system can terminate the process <NUM> and can assign a default private access value (e.g., the private access value M<NUM>) to the nodes system did not assign private access values during the iterations. Otherwise, the selection system can proceed to the next iteration of the process <NUM>. In implementations where the selection system identifies the set of sub-trees for the next iteration, the selection system can provide the identified set of sub-trees for use in the next iteration.

When the selection of private access values is complete, the selection system can return the private access values assigned to the nodes of the model tree <NUM> (step <NUM>).

<FIG> is a flow diagram of an example process for classifying the access values of nodes. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, a selection system, e.g., the selection system <NUM> of <FIG>, appropriately programmed in accordance with this specification, can perform the process <NUM>.

The classification system receives a sub-tree and a current threshold (step <NUM>).

If the root node of the sub-tree has an access value below the current threshold, the classification system can immediately classify all of the nodes of the sub-tree as not satisfying the acceptance criterion.

The classification system can classify the nodes in the sub-tree following a sequence of iterations processing each level of the sub-tree, starting from the deepest level of the sub-tree and continuing to the level of the root node of the sub-tree.

For each level of the sub-tree, the classification system can determine whether the level includes at least one unclassified node having an access value that exceeds the current threshold value (step <NUM>). In some implementations, the classification system can determine whether the level has an unclassified node with an access value exceeding the current threshold using sparse vector techniques, e.g. the sparse vector technique presented as Algorithm <NUM> by <NPL>. If the level has no unclassified nodes with access values exceeding the current threshold, the classification system can classify all of the nodes at the level as not satisfying the acceptance criterion and immediately begin the iteration for the next level.

The classification system transforms the access value of the nodes at the current level by adding noise to the access value (step <NUM>). In some implementations, the classification system can add noise to a given access value by sampling a noise value from a particular noise distribution and adding the noise value to the access value. The particular noise distribution can be any suitable distribution. For example, the particular noise distribution can be a Gaussian distribution. As another example, the particular noise distribution can be a Laplace distribution. As another example, the particular noise distribution can be a truncated Laplace distribution. As a further example, the particular noise distribution can be a truncated Laplace distribution having parameter <MAT> and range Ri.

In some implementations, the classification system can further transform the access values of the nodes by adding one or more offsets to the access values. For example, during the iteration of the process <NUM> corresponding to the private access value Mi, the classification system can add an offset Δi to the access values, where the system determines Δi following: <MAT>.

As another example, during the iteration of the process <NUM> corresponding to the private access value Mi, the classification system can add the range Ri to the access values.

The classification system can classify the nodes at the current level based on whether the transformed access values of the node exceed the current threshold (<NUM>). When the classification system classifies a particular node as satisfying the acceptance criterion, the system can immediately classify all of the ancestor nodes of the particular node as also satisfying the acceptance criterion.

The classification system can then finish the classification if there are no unclassified nodes remaining in the tree (step <NUM>). If there are still unclassified nodes remaining in the tree, the classification system can continue with the iteration for the next level of the tree.

When the classification finishes, the classification system can return the classifications of the nodes of the received sub-tree (step <NUM>).

<FIG> is a flow diagram of an example process for generating a resource access estimate. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, an estimation system, e.g., the estimation system <NUM> of <FIG>, appropriately programmed in accordance with this specification, can perform the process <NUM>.

The estimation system receives a tree model and associated private access values (step <NUM>).

The estimation system can process the query request to estimate the number of users that accessed the digital resource in a particular time window and produce a numerical representation of the particular query time window (step <NUM>).

In some implementations, the estimation system can identify the nodes of the tree that are relevant (i.e. satisfy a selection criterion) to the query request by processing the query keys associated to each node (step <NUM>). A node can be relevant to the query (i.e., satisfy the selection criterion) if it is assigned a query key characterizing the node as falling within the time window specified by the query request. The estimation system can determine which nodes are relevant to the numerical representation of the query time window by processing the numerical representation of the query time window alongside the query keys assigned to the nodes. An example algorithm for determining which nodes are relevant to the query is presented by <NPL>.

The estimation system then returns an estimation for the query request based on the private access values assigned to the nodes of the tree relevant to the query request (step <NUM>). In some implementations, the estimation can be based on a combination of the private access values assigned to nodes relevant to the request. In some implementations, the estimation can be a sum of private access values assigned to nodes relevant to the request. In some implementations, the relevant nodes can be limited to the relevant leaf nodes of the tree model.

<FIG> is a block diagram of an example environment <NUM> in which digital components are transmitted for presentation with electronic documents.

The example environment <NUM> includes a network <NUM>, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. The network <NUM> connects electronic document servers <NUM>, client devices <NUM>, digital component servers <NUM>, and a digital component distribution system <NUM> (also referred to as a "distribution system" <NUM>). The example environment <NUM> may include many different electronic document servers <NUM>, client devices <NUM>, and digital component servers <NUM>.

A client device <NUM> is an electronic device that is capable of requesting and receiving resources over the network <NUM>. Example client devices <NUM> include personal computers, mobile communication devices (e.g., mobile phones), and other devices that can send and receive data over the network <NUM>. A client device <NUM> typically includes a user application, such as a web browser, to facilitate the sending and receiving of data over the network <NUM>, but native applications executed by the client device <NUM> can also facilitate the sending and receiving of data over the network <NUM>.

An electronic document is data that presents a set of content at a client device <NUM>. Examples of electronic documents include webpages, word processing documents, portable document format (PDF) documents, images, videos, search results pages, and feed sources. Native applications (e.g., "apps"), such as applications installed on mobile, tablet, or desktop computing devices are also examples of electronic documents. Electronic documents can be provided to client devices <NUM> by electronic document servers <NUM> ("Electronic Doc Servers"). For example, the electronic document servers <NUM> can include servers that host publisher websites. In this example, the client device <NUM> can initiate a request for a given publisher webpage, and the electronic server <NUM> that hosts the given publisher webpage can respond to the request by sending machine executable instructions that initiate presentation of the given webpage at the client device <NUM>.

In another example, the electronic document servers <NUM> can include app servers from which client devices <NUM> can download apps. In this example, the client device <NUM> can download files required to install an app at the client device <NUM>, and then execute the downloaded app locally.

Electronic documents can include a variety of content. For example, an electronic document can include static content (e.g., text or other specified content) that is within the electronic document itself and/or does not change over time. Electronic documents can also include dynamic content that may change over time or on a per-request basis. For example, a publisher of a given electronic document can maintain a data source that is used to populate portions of the electronic document. In this example, the given electronic document can include one or more tags or scripts that cause the client device <NUM> to request content from the data source when the given electronic document is processed (e.g., rendered or executed) by a client device <NUM>. The client device <NUM> integrates the content obtained from the data source into the given electronic document to create a composite electronic document including the content obtained from the data source.

In some situations, a given electronic document can include one or more digital component tags or digital component scripts that reference the digital component distribution system <NUM>. In these situations, the digital component tags or digital component scripts are executed by the client device <NUM> when the given electronic document is processed by the client device <NUM>. Execution of the digital component tags or digital component scripts configures the client device <NUM> to generate a request for one or more digital components <NUM> (referred to as a "component request"), which is transmitted over the network <NUM> to the digital component distribution system <NUM>. For example, a digital component tag or digital component script can enable the client device <NUM> to generate a packetized data request including a header and payload data. The component request <NUM> can include event data specifying features such as a name (or network location) of a server from which the digital component is being requested, a name (or network location) of the requesting device (e.g., the client device <NUM>), and/or information that the digital component distribution system <NUM> can use to select one or more digital components provided in response to the request. The component request <NUM> is transmitted, by the client device <NUM>, over the network <NUM> (e.g., a telecommunications network) to a server of the digital component distribution system <NUM>.

The component request <NUM> can include event data specifying other event features, such as the electronic document being requested and characteristics of locations of the electronic document at which digital component can be presented. For example, event data specifying a reference (e.g., URL) to an electronic document (e.g., webpage) in which the digital component will be presented, available locations of the electronic documents that are available to present digital components, sizes of the available locations, and/or media types that are eligible for presentation in the locations can be provided to the digital component distribution system <NUM>. Similarly, event data specifying keywords associated with the electronic document ("document keywords") or entities (e.g., people, places, or things) that are referenced by the electronic document can also be included in the component request <NUM> (e.g., as payload data) and provided to the digital component distribution system <NUM> to facilitate identification of digital components that are eligible for presentation with the electronic document. The event data can also include a search query that was submitted from the client device <NUM> to obtain a search results page, and/or data specifying search results and/or textual, audible, or other visual content that is included in the search results.

Component requests <NUM> can also include event data related to other information, such as information that a user of the client device has provided, geographic information indicating a state or region from which the component request was submitted, or other information that provides context for the environment in which the digital component will be displayed (e.g., a time of day of the component request, a day of the week of the component request, a type of device at which the digital component will be displayed, such as a mobile device or tablet device). Component requests <NUM> can be transmitted, for example, over a packetized network, and the component requests <NUM> themselves can be formatted as packetized data having a header and payload data. The header can specify a destination of the packet and the payload data can include any of the information discussed above.

The component distribution system <NUM> chooses digital components that will be presented with the given electronic document in response to receiving the component request <NUM> and/or using information included in the component request <NUM>. In some implementations, a digital component is selected (using the techniques described herein) in less than a second to avoid errors that could be caused by delayed selection of the digital component. For example, delays in providing digital components in response to a component request <NUM> can result in page load errors at the client device <NUM> or cause portions of the electronic document to remain unpopulated even after other portions of the electronic document are presented at the client device <NUM>. Also, as the delay in providing the digital component to the client device <NUM> increases, it is more likely that the electronic document will no longer be presented at the client device <NUM> when the digital component is delivered to the client device <NUM>, thereby negatively impacting a user's experience with the electronic document. Further, delays in providing the digital component can result in a failed delivery of the digital component, for example, if the electronic document is no longer presented at the client device <NUM> when the digital component is provided.

In some implementations, the digital component distribution system <NUM> is implemented in a distributed computing system that includes, for example, a server and a set of multiple computing devices <NUM> that are interconnected and identify and distribute digital components in response to requests <NUM>. The set of multiple computing devices <NUM> operate together to identify a set of digital components that are eligible to be presented in the electronic document from a corpus of millions of available digital components (DC1-x). The millions of available digital components can be indexed, for example, in a digital component database <NUM>. Each digital component index entry can reference the corresponding digital component and/or include distribution parameters (DP1-DPx) that contribute to (e.g., condition or limit) the distribution/transmission of the corresponding digital component. For example, the distribution parameters can contribute to the transmission of a digital component by requiring that a component request include at least one criterion that matches (e.g., either exactly or with some pre-specified level of similarity) one of the distribution parameters of the digital component.

In some implementations, the distribution parameters for a particular digital component can include distribution keywords that must be matched (e.g., by electronic documents, document keywords, or terms specified in the component request <NUM>) in order for the digital component to be eligible for presentation. In other words, the distribution parameters are used to trigger distribution (e.g., transmission) of the digital components over the network <NUM>. The distribution parameters can also require that the component request <NUM> include information specifying a particular geographic region (e.g., country or state) and/or information specifying that the component request <NUM> originated at a particular type of client device (e.g., mobile device or tablet device) in order for the digital component to be eligible for presentation.

The distribution parameters can also specify an eligibility value (e.g., ranking score, bid, or some other specified value) that is used for evaluating the eligibility of the digital component for distribution/transmission (e.g., among other available digital components), for example, by the component evaluation process. In some situations, the eligibility value can specify a maximum amount of compensation that a provider of the digital component is willing to submit in response to the transmission of the digital component (e.g., for each instance of specific events attributed to the presentation of the digital component, such as user interaction with the digital component).

The identification of the eligible digital component can be segmented into multiple tasks 817a-817c that are then assigned among computing devices within the set of multiple computing devices <NUM>. For example, different computing devices in the set <NUM> can each analyze a different portion of the digital component database <NUM> to identify various digital components having distribution parameters that match information included in the component request <NUM>. In some implementations, each given computing device in the set <NUM> can analyze a different data dimension (or set of dimensions) and pass (e.g., transmit) results (Res <NUM>-Res <NUM>) 818a-818c of the analysis back to the digital component distribution system <NUM>. For example, the results 818a818c provided by each of the computing devices in the set <NUM> may identify a subset of digital components that are eligible for distribution in response to the component request and/or a subset of the digital components that have certain distribution parameters. The identification of the subset of digital components can include, for example, comparing the event data to the distribution parameters, and identifying the subset of digital components having distribution parameters that match at least some features of the event data.

The digital component distribution system <NUM> aggregates the results 818a-818c received from the set of multiple computing devices <NUM> and uses information associated with the aggregated results to: (i) select one or more digital components that will be provided in response to the request <NUM>, and (ii) determine transmission requirements for the one or more digital components. For example, the digital component distribution system <NUM> can select a set of winning digital components (one or more digital components) based on the outcome of one or more component evaluation processes. In turn, the digital component distribution system <NUM> can generate and transmit, over the network <NUM>, reply data <NUM> (e.g., digital data representing a reply) that enables the client device <NUM> to integrate the set of winning digital components into the given electronic document, such that the set of winning digital components and the content of the electronic document are presented together at a display of the client device <NUM>.

In some implementations, the client device <NUM> executes instructions included in the reply data <NUM>, which configures and enables the client device <NUM> to obtain the set of winning digital components from one or more digital component servers. For example, the instructions in the reply data <NUM> can include a network location (e.g., a Uniform Resource Locator (URL)) and a script that causes the client device <NUM> to transmit a server request (SR) <NUM> to the digital component server <NUM> to obtain a given winning digital component from the digital component server <NUM>. In response to the request, the digital component server <NUM> will identify the given winning digital component specified in the server request <NUM> (e.g., within a database storing multiple digital components) and transmit, to the client device <NUM>, digital component data (DC Data) <NUM> that presents the given winning digital component in the electronic document at the client device <NUM>.

To facilitate searching of electronic documents, the environment <NUM> can include a search system <NUM> that identifies the electronic documents by crawling and indexing the electronic documents (e.g., indexed based on the crawled content of the electronic documents). Data about the electronic documents can be indexed based on the electronic document with which the data are associated. The indexed and, optionally, cached copies of the electronic documents are stored in a search index <NUM> (e.g., hardware memory device(s)). Data that are associated with an electronic document is data that represents content included in the electronic document and/or metadata for the electronic document.

Client devices <NUM> can submit search queries to the search system <NUM> over the network <NUM>. In response, the search system <NUM> accesses the search index <NUM> to identifiy electronic documents that are relevant to the search query. The search system <NUM> identifies the electronic documents in the form of search results and returns the search results to the client device <NUM> in a search results page. A search result is data generated by the search system <NUM> that identifies an electronic document that is responsive (e.g., relevant) to a particular search query, and includes an active link (e.g., hypertext link) that causes a client device to request data from a specified network location (e.g., URL) in response to user interaction with the search result. An example search result can include a web page title, a snippet of text or a portion of an image extracted from the web page, and the URL of the web page. Another example search result can include a title of a downloadable application, a snippet of text describing the downloadable application, an image depicting a user interface of the downloadable application, and/or a URL to a location from which the application can be downloaded to the client device <NUM>. In some situations, the search system <NUM> can be part of, or interact with, an application store (or an online portal) from which applications can be downloaded for install at a client device <NUM> in order to present information about downloadable applications that are relevant to a submitted search query. Like other electronic documents, search results pages can include one or more slots in which digital components (e.g., advertisements, video clips, audio clips, images, or other digital components) can be presented.

To select a digital component to be transmitted in response to a component request, the distribution system <NUM> may identify a set of digital components that are eligible to be transmitted in response to the component request. The distribution system <NUM> may then select one or more of the eligible digital components to be transmitted through, e.g., an auction procedure. In some implementations, the distribution system <NUM> performs an auction procedure by ranking the eligible digital components in accordance with their respective eligibility values, and selecting one or more highest-ranked digital components to be transmitted in response to the component request.

For example, the distribution system <NUM> may identify digital components A, B, and C as eligible to be transmitted in response to a component request. In this example, digital component A has an eligibility value of <NUM>, digital component B has an eligibility value of <NUM>, and digital component C has an eligibility value of5. <NUM>, where the eligibility values of the digital components represent bids associated with the digital components. The distribution system <NUM> may rank (e.g., in descending order) the digital components in accordance with their respective eligibility values as: C, A, B. Finally, the distribution system <NUM> may select the highest ranked digital component C for transmission in response to the component request.

After selecting a digital component to be transmitted in response to a digital component request, the distribution system <NUM> determines a transmission requirement for the selected digital component. A transmission requirement specifies an action to be performed by the provider of a digital component in response to a transmission of the digital component. For example, the transmission requirement may specify that the provider of the digital component submit an amount of compensation in response to the transmission of the digital component. In some cases, the amount of compensation specifies an amount to be submitted for each instance of specific events attributed to the presentation of the digital component (e.g., user interactions with the digital component).

The distribution system <NUM> may determine the transmission requirement of the selected digital component based on the eligibility value of the selected digital component and/or the eligibility values of the other digital components that were determined as eligible to be transmitted in response to the component request. For example, the distribution system <NUM> may identify digital components A, B, and C as eligible for transmission in response to a digital component request, where A, B, and C have respective eligibility values of <NUM>, <NUM>, and <NUM>. The distribution system <NUM> may select digital component C for transmission (since it has the highest eligibility value), and may determine the transmission requirement for digital component C to be the next highest eligibility value from amongst the eligibility values of the eligible digital components. In this example, next highest eligibility value is $<NUM> (i.e., the eligibility value of digital component A), and therefore the distribution system <NUM> may determine the transmission requirement of digital component C to be $<NUM>.

As described above, the distribution system <NUM> may identify a set of digital components that are eligible to be transmitted for presentation in an electronic document in response to a digital component request based on distribution parameters corresponding to each digital component. In some cases, the distribution parameters corresponding to a digital component may include a keyword cluster (i.e., a set of multiple keywords). The distribution system <NUM> may determine that one or more keywords from the keyword cluster must be matched (e.g., by electronic documents, document keywords, or terms specified in a digital component request) in order for the digital component to be eligible for transmission.

For example, the distribution system <NUM> may receive a digital component request which includes a specific keyword. In this example, the distribution system <NUM> may determine that a particular digital component with distribution parameters specifying a keyword cluster is eligible for transmission in response to the digital component request only if the specific keyword is included in the keyword cluster.

In some implementations, the distribution system <NUM> enables providers of digital components to set distribution parameters specifying keyword clusters from a predetermined set of keyword clusters. The keyword clusters output by the clustering system <NUM> may define a grouping of keywords into semantically related keyword clusters. For example, a keyword cluster can define an assignment of the keywords "shoes", "shoe", "footwear", "boots", "cleats", "heels", "slippers", "sneakers", and the like, to the same cluster.

Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible storage medium which may be non-transitory for execution by, or to control the operation of, data processing apparatus.

Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

Claim 1:
A method performed by one or more computers, the method comprising:
obtaining (<NUM>) access data for a digital resource, wherein the access data comprises, for each time point in a sequence of time points, data identifying a set of users that accessed the digital resource at the time point;
generating (<NUM>) a tree model based on the access data, wherein each node of the tree model is associated with a respective access value that characterizes a number of users that satisfy a node-specific selection criterion based on accessing the digital resource;
selecting (<NUM>), for each node in the tree model, a respective private access value for the node that: (i) is an approximation of the access value for the node, and (ii) is selected from a finite set of possible private access values;
receiving (<NUM>) a request to determine a number of users that accessed the digital resource at least a predefined number of times within a time window; and
in response to the request:
generating (<NUM>) an estimate for the number of users that accessed the digital resource at least the predefined number of times within the time window based on private access values associated with one or more nodes in the tree model.