Patent Description:
<CIT> describes systems and methods of measurement and modification of advertisements and content. In one example, advertisements/content items (or web servers or applications that present the advertisements/content items) send signals to a measurement server in response to certain events or actions. The signals identify the advertisement/content item and the user that caused the event or performed the action. The measurement server aggregates received signals from different advertisements/content items to determine metrics such as digital brand lift (e.g., a change in brand awareness due to an advertisement/advertising campaign). The measurement server can send computed information back to an advertisement/content item, so that the advertisement/content item (or web server/application) can self-modify and/or deploy additional advertisements/content items. <CIT> Al discloses a system configured to use one or more algorithms or computational models to enable one or more participants of a personalized advertising chain to anonymously determine a user's preferences (or otherwise provide secured personal data analysis) by analyzing encrypted personal data.

In a first aspect, the invention claimed in this European patent is defined by a method according to claim <NUM>. Optional features are set out in claims <NUM> to <NUM>.

In a second aspect, the invention claimed in this European patent is defined by a system according to claim <NUM>. Optional features are set out in claims <NUM> to <NUM>.

In a third aspect, the invention claimed in this European patent is defined by a computer-readable medium according to claim <NUM>. Optional features are set out in claims <NUM> and <NUM>.

In some implementations, the interaction measurement SDK bundles interaction data from a number of events and transmits the bundled interaction data in response to a threshold. In some implementations, the threshold is a threshold period of time or a threshold number of events. In some implementations, the interaction SDK generates a signature of the interaction data using a hashing algorithm, and wherein the first computing device validates an authenticity of the interaction data using the signature. In some implementations, the method further includes securely transmitting the interaction data to a second computing device associated with the mobile application, and wherein the interaction SDK removes personal identifiers from the interaction data prior to securely transmitting the interaction data to the first computing device. In some implementations, the interaction data includes an intermediary identifier, and wherein method further includes identifying, by the second computing device, supplemental data using the intermediary identifier, transmitting, by the second computing device to the first computing device, the supplemental data, and correlating, by the first computing device, the interaction data with the supplemental data. In some implementations, the first computing device does not have access to personally identifying information. In some implementations, the method further includes analyzing, by the first computing device, at least the interaction data to generate data describing usage of the mobile application in real time. In some implementations, analyzing at least the interaction data includes identifying a previous interaction of the user that was not performed using the mobile application.

In some implementations, wherein the instructions cause the one or more processors to bundle interaction data from a number of events and transmit the bundled interaction data in response to a threshold. In some implementations, the threshold is a threshold period of time or a threshold number of events. In some implementations, the instructions cause the one or more processors to generate a signature of the interaction data using a hashing algorithm, and wherein the first computing device validates an authenticity of the interaction data using the signature. In some implementations, the instructions further cause the one or more processors to securely transmit the interaction data to the second computing device and remove personal identifiers from the interaction data prior to securely transmitting the interaction data to the first computing device. In some implementations, wherein the interaction data includes an intermediary identifier, wherein the second computing device identifies supplemental data using the intermediary identifier, wherein second computing device transmits the supplemental data to the first computing device, and wherein the first computing device correlates the interaction data with the supplemental data. In some implementations, the first computing device does not have access to personally identifying information. In some implementations, the first computing device analyzes at least the interaction data to generate data describing usage of the mobile application in real time. In some implementations, analyzing at least the interaction data includes identifying a previous interaction of the user that was not performed using the mobile application.

In some implementations, wherein the SDK bundles interaction data from a number of events and transmits the bundled interaction data in response to a threshold, wherein the threshold is a threshold period of time or a threshold number of events, wherein the interaction SDK generates a signature of the interaction data using a hashing algorithm, and wherein the first computing device validates an authenticity of the interaction data using the signature. In some implementations, the instructions further cause the one or more processors to securely transmit the interaction data to a second computing device and remove personal identifiers from the interaction data prior to securely transmitting the interaction data to the first computing device.

The various aspects and implementations may be combined where appropriate.

Following below are more detailed descriptions of various concepts related to, and implementations of, methods, apparatuses, and systems for collecting event data in a timely and redundant manner while preserving privacy. The various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the described concepts are not limited to any particular manner of implementation.

In many domains, it may be desirable to collect event data from mobile devices. For example, in the event of a malfunction (e.g., a software crash, a hardware failure, etc.), it is often desirable to collect event data such as device settings, operating parameters, and the like in order to diagnose and fix the malfunction. In other domains, it may be desirable for a provider of third party content to determine a rate of interaction with that third party content.

System and methods of the present disclosure relate generally to reporting mobile device event data. More specifically, systems and methods of the present disclosure relate to a unique software development kit (SDK) and computer architecture methodologies to collect and correlate event data from mobile devices.

Typically, event data from mobile devices is sent to a single endpoint (e.g., an aggregator, etc.) and then forwarded to an analysis system for analysis (e.g., correlation, etc.). For example, event data generated by a first mobile application may be transmitted to servers controlled by a developer of the first mobile application which may collect the data and forward it to an analysis system which determines aggregate statistics associated with the event data. However, the single endpoint may not always be available. For example, the endpoint may experience unexpected failures or routine maintenance which prevent it from receiving data from mobile devices and/or forwarding data to the analysis system. It is desirable to have a robust architecture that facilitates analysis of event data even if the first endpoint becomes unavailable. Therefore, there is a need for a unique SDK and computer architecture methodology to facilitate transmission of event data directly to the analysis system.

As discussed above, event data from mobile devices is generally sent to a single endpoint (e.g., an aggregator, etc.) and then forwarded to an analysis system for analysis. For example, a developer may receive event data and send the event data to a third party for analysis (e.g., because analyzing the event data is prohibitively difficult, complex, and/or expensive for the developer, etc.). However, transmitting event data from a client device to an aggregator and then to an analysis system introduces a time delay. It may be undesirable to introduce a time delay. For example, an analysis system may be used to identify bugs in application <NUM>, such as link rot. The time delay introduced by forwarding data from client devices to an aggregator and then to the analysis system may use additional system and network resources which could be more efficiently deployed elsewhere and may increase the amount of time it takes to identify a bug in application <NUM>. For example, the analysis system may detect a stale hyperlink several days after it goes stale (e.g., ceases to function properly, etc.). Therefore, there is a need for a unique SDK and computer architecture methodology to facilitate timely analysis of application events.

Aspects of the present disclosure provide improved data reporting and analysis pathways and computer architectures. The pathways and architectures may be used to report and analyze mobile device event data in a robust and timely manner that prevents exposing PII to third parties.

To ensure robustness against endpoint failure, systems and methods of the present disclosure introduce an SDK that facilitates sending data to a second endpoint. A non-limiting example implementation is as follows: a user using a desktop device may view a first content item. A first computing system may register that the user viewed the first content item. The user using a mobile application may then interact with a second content item, thereby causing the mobile application to transmit event data associated with the interaction to the first computing device. A SDK embedded in the mobile application may register that the user interacted with the second content item and may prepare event data associated with the interaction for transmission. For example, the event data may include the content item, a time, and a source of the content item (e.g., a website, etc.). The SDK may remove any personal identifiers from the event data, such as a mobile device identifier. The SDK may then securely transmit the event data to a second computing device. The first computing device may identify the user and prepare supplemental data associated with the user's interaction with the second content item. For example, the first computing device may identify data corresponding to the user's interaction with the first content item. The first computing device may remove any personal identifiers from the supplemental data and transmit the supplemental data to the second computing device. The second computing device may correlate the event data with the supplemental data to determine a result.

In various embodiments, the SDK of the present disclosure removes identifiers from event data before transmitting the event data to the analysis system. In various embodiments, the SDK of the present disclosure facilitates correlation of data through intermediary identifiers. For example, the SDK of the present disclosure may transmit event data having an intermediary identifier (e.g., an event identifier, etc.) to a first endpoint (e.g., an aggregator, etc.) and the analysis system. The first endpoint may be associated with an application (e.g., a server maintained by the developers of the application, etc.). In some implementations, the first endpoint may independently maintain user information such as account balances associated with user accounts of the application. The first endpoint may identify supplemental data. For example, the first endpoint may maintain a database having interaction data and may search the database for interaction data associated with a user. The supplemental data may include an intermediary identifier. The first endpoint may transmit the supplemental data to the analysis system which may correlate the supplemental data with the event data using the intermediary identifier, thereby preserving PII.

Referring now to <FIG>, a system <NUM> for collecting event data from mobile devices is shown, according to an illustrative implementation. System <NUM> includes client device <NUM>, first data processing system <NUM>, and second data processing system <NUM>. In various implementations, components of system <NUM> communicate over network <NUM>. Network <NUM> may include computer networks such as the Internet, local, wide, metro or other area networks, intranets, satellite networks, other computer networks such as voice or data mobile phone communication networks, combinations thereof, or any other type of electronic communications network. Network <NUM> may include or constitute a display network (e.g., a subset of information resources available on the Internet that are associated with a content placement or search engine results system, or that are eligible to include third party content items as part of a content item placement campaign). In various implementations, network <NUM> facilitates secure communication between components of system <NUM>. As a non-limiting example, network <NUM> may implement transport layer security (TLS), secure sockets layer (SSL), hypertext transfer protocol secure (HTTPS), and/or any other secure communication protocol.

Client device <NUM> may be a mobile computing device, smartphone, tablet, or any other device configured to facilitate receiving, displaying, and interacting with content (e.g., web pages, mobile applications, etc.). Client device <NUM> may include an application <NUM> to receive and display online content and to receive user interaction with the online content. For example, application <NUM> may be a web browser. Additionally or alternatively, application <NUM> may be a mobile application associated with a specific merchant.

In various implementations, application <NUM> interacts with a content publisher to receive online content. For example, application <NUM> may receive an information resource from a content publisher. The information resources may include web-based content items such as a web page or other online documents. The information resources may include instructions (e.g., scripts, executable code, etc.) that when interpreted by application <NUM> cause application <NUM> to display a graphical user interface such as an interactable web page to a user.

Application <NUM> is shown to include software development kit <NUM> having event circuit <NUM>, bundling circuit <NUM>, encryption circuit <NUM>, and reporting circuit <NUM>. Software development kit (SDK) <NUM> may include a collection of software development tools contained in a package. SDK <NUM> may include an application programming interface (API). In some implementations, SDK <NUM> includes one or more libraries having reusable functions that interface with a particular system software (e.g., iOS, Android, etc.). SDK <NUM> may facilitate embedding functionality in application <NUM>. For example, a developer may use SDK <NUM> to automatically transmit event data whenever an event of a specific type occurs on application <NUM>. As a further example, SDK <NUM> may include a reusable function configured to collect and report device analytics and a developer may insert the reusable function into the instructions of application <NUM> to cause the reusable function to be called during specific actions of application <NUM>. In some implementations, event circuit <NUM>, building circuit <NUM>, encryption circuit <NUM>, and/or reporting circuit <NUM> are functionalities provided by SDK <NUM> (e.g., reusable functions, etc.).

Event circuit <NUM> may detect events within application <NUM>. In various implementations, event circuit <NUM> may be configured to trigger other functionality based on detecting specific events (e.g., transactions, in-app purchases, achieving a certain level in an in-app game, performing a certain number of actions, spending a certain amount of time interacting with an application, etc.). For example, event circuit <NUM> may trigger bundling circuit <NUM> upon detecting an event within application <NUM>. In various implementations, SDK <NUM> includes a function that is embedded in application <NUM> to trigger event circuit <NUM>. For example, a developer may include a function of SKD <NUM> in a transaction confirmation functionality of application <NUM> that causes event circuit <NUM> to detect a confirmed transaction. It should be understood that events may include any action important to a developer within an application and are not limited to the examples expressly contemplated herein. In various implementations, event circuit <NUM> is configured to differentiate between different types of events. For example, event circuit <NUM> may trigger a first set of actions based on a first type of detected event and may trigger a second set of actions based on a second type of detected event. In various implementations, event circuit <NUM> is configured to collect event data associated with the detected event and transmit the collected event data to bundling circuit <NUM>.

Bundling circuit <NUM> bundles (e.g., aggregate, etc.) event data. In various implementations, bundling circuit <NUM> receives event data associated with a detected event from event circuit <NUM>. Bundling circuit <NUM> may collect event data from one or more events and bundle the event data for transmission. For example, bundling circuit <NUM> may collect event data from ten events and combine the event data into a single bundle. Event data may include a timestamp of the event, a name of the event, and/or parameters of the event (e.g., a purchased item, a price, a currency, discounts, subscription information, etc.). In some implementations, bundling circuit <NUM> transmits bundles to encryption circuit <NUM>. Additionally or alternatively, bundling circuit <NUM> may transmit bundles to reporting circuit <NUM>. In various implementations, bundling circuit <NUM> generates a data bundle. The data bundle may include a bundle index, a bundle timestamp, bundle data, and a bundle signature. In various implementations, the bundle signature is generated by encryption circuit <NUM>, discussed below. In some implementations, the bundle index specifies where specific event data is located in the bundle data. For example, the bundle data may include a byte array and the bundle index may include an index to the byte array.

Encryption circuit <NUM> may encrypt data to produce encrypted data. For example, encryption circuit <NUM> may encrypt bundled event data. Additionally or alternatively, encryption circuit <NUM> may perform various obliviating functions on received data. For example, encryption circuit <NUM> may remove identifiers (e.g., IP address, device identifiers, etc.), fragment event data, add noise, or perform other functions to anonymize data. In various implementations, encryption circuit <NUM> implements asymmetric encryption. For example, encryption circuit <NUM> may implement a Rivest-Shamir-Adleman (RSA) cryptosystem. In various implementations, encryption circuit <NUM> may receive a public key from first data processing system <NUM> and use the public key to encrypt received data. Additionally or alternatively, encryption circuit <NUM> may generate a signature associated with received data. For example, encryption circuit <NUM> may generate a hash of bundled event data received from bundling circuit <NUM>. One or more hashing functions may be used. For example, encryption circuit <NUM> may implement a SHA-<NUM>, Scrypt, Balloon, and/or Argon2 hashing function.

Reporting circuit <NUM> may transmit bundled event data to first data processing system <NUM> and/or second data processing system <NUM>. In various implementations, reporting circuit <NUM> transmits data via network <NUM>. Reporting circuit <NUM> may confirm the transmission of data. For example, reporting circuit <NUM> may transmit bundled event data to first data processing system <NUM> and receive a confirmation that the bundled event data was received successfully. In some implementations, reporting circuit <NUM> first attempts to transmit bundled event data to first data processing system <NUM> and if that fails then attempts to transmit bundled event data to second data processing system <NUM>. Additionally or alternatively, reporting circuit <NUM> may transmit data to first data processing system <NUM> and second data processing system <NUM> in parallel. In some implementations, reporting circuit <NUM> transmits different data to first data processing system <NUM> and second data processing system <NUM>. For example, reporting circuit <NUM> may transmit encrypted event data having device identifiers to first data processing system <NUM> and may transmit unencrypted event data without device identifiers to second data processing system <NUM>. Additionally or alternatively, reporting circuit <NUM> may transmit the same data to first data processing system <NUM> and second data processing system <NUM>.

In various implementations, reporting circuit <NUM> transmits data periodically. For example, reporting circuit <NUM> may transmit data at a predefined time. As another example, reporting circuit <NUM> may transmit data on an interval (e.g., every ten minutes, every ten hours, etc.). Additionally or alternatively, reporting circuit <NUM> may transmit data in response to a threshold. For example, reporting circuit <NUM> may transmit data in response to bundling circuit receiving a threshold number of event data from events (e.g., ten events, one-hundred events, etc.). In some implementations, reporting circuit <NUM> transmits data dynamically. For example, reporting circuit <NUM> may transmit data in response to client device <NUM> being connected to a charging source. As a further example, reporting circuit <NUM> may transmit data in response to the transmission bundle reaching a specified data size.

In various implementations, reporting circuit <NUM> reports metrics. For example, reporting circuit <NUM> may transmit metrics alongside each data bundle. The metrics may include a size of the data bundle, a timestamp of the transmission and/or generation of the data bundle, a data bundle index, an SDK identifier, and/or a signature of the data bundle. In various implementations, the SDK identifier includes information associated with SDK <NUM>. For example, the SDK identifier may include a version number of SDK <NUM>. The signature of the data bundle may include a hash of the data bundle contents as discussed above with reference to encryption circuit <NUM>.

First data processing system <NUM> may receive event data from SDK <NUM> and/or generate supplemental data. In various implementations, first data processing system <NUM> receives a data bundle from SDK <NUM>, identifies supplemental data based on the contents of the data bundle, and transmits the supplemental data to second data processing system <NUM>. In some implementations, first data processing system <NUM> anonymizes the supplemental data before transmission to second data processing system <NUM>. For example, first data processing system <NUM> may remove personal identifiers from the supplemental data before transmission to second data processing system <NUM>. First data processing system <NUM> may be a server, distributed processing cluster, cloud processing system, or any other computing device. First data processing system <NUM> may include or execute at least one computer program or at least one script. In some implementations, first data processing system <NUM> includes combinations of software and hardware, such as one or more processors configured to execute one or more scripts.

First data processing system <NUM> is shown to include database <NUM> and processing circuit <NUM>. Database <NUM> may store supplemental data. For example, database <NUM> may include information associated with previous content interactions. As an additional example, a user using a desktop computer may navigate to a website and interact with a content item. First data processing system <NUM> may receive information associated with the user's interaction with the content item and store the information in database <NUM>. The supplemental data may include content identifiers, device identifiers, user identifiers, clickstreams, and/or the like. Database <NUM> may include one or more storage mediums. The storage mediums may include but are not limited to magnetic storage, optical storage, flash storage, and/or RAM. First data processing system <NUM> may implement or facilitate various APIs to perform database functions (i.e., managing data stored in database <NUM>). The APIs can be but are not limited to SQL, ODBC, JDBC, and/or any other data storage and manipulation API.

Processing circuit <NUM> may include processor <NUM> and memory <NUM>. Memory <NUM> may have instructions stored thereon that, when executed by processor <NUM>, cause processing circuit <NUM> to perform the various operations described herein. The operations described herein may be implemented using software, hardware, or a combination thereof. Processor <NUM> may include a microprocessor, ASIC, FPGA, etc., or combinations thereof. In many implementations, processor <NUM> may be a multi-core processor or an array of processors. Memory <NUM> may include, but is not limited to, electronic, optical, magnetic, or any other storage devices capable of providing processor <NUM> with program instructions. Memory <NUM> may include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, EEPROM, EPROM, flash memory, optical media, or any other suitable memory from which processor <NUM> can read instructions. The instructions may include code from any suitable computer programming language such as, but not limited to, C, C++, C#, Java, JavaScript, Perl, HTML, XML, Python and Visual Basic.

Memory <NUM> may include data aggregation circuit <NUM>. Data aggregation circuit <NUM> may identify supplemental data from database <NUM>. In various embodiments, data aggregation circuit <NUM> receives a data bundle from SDK <NUM> and analyzes contents of the data bundle to identify supplemental data. For example, aggregation circuit <NUM> may receive a data bundle including an event associated with a first device identifier, may identify supplemental data including previous content interactions associated with the device identifier, and may transmit the supplemental data to second data processing system <NUM>. In various implementations, supplemental data is associated with contents of a data bundle. For example, the supplemental data may be related to event data of an event included in a data bundle through an intermediate identifier. In some implementations, data aggregation circuit <NUM> verifies the contents of a received data bundle. For example, data aggregation circuit <NUM> may verify a signature included in the data bundle. Additionally or alternatively, data aggregation circuit <NUM> may decrypt the contents of the data bundle. For example, data aggregation circuit <NUM> may receive an asymmetrically encrypted data bundle and decrypt the asymmetrically encrypted data bundle using a private key.

Second data processing system <NUM> may receive event data from SDK <NUM> and facilitate performing analysis on received data to generate information. For example, second data processing system <NUM> may receive a data bundle including event data from SDK <NUM> and supplemental data from first data processing system <NUM> and securely correlate the received data to generate information. As another example, second data processing system <NUM> may receive first data associated with a transaction from SDK <NUM> and second data associated with a user interaction with a content item from first data processing system <NUM> and correlate the first and second data.

In various embodiments, second data processing system <NUM> generates aggregate information. For example, second data processing system <NUM> may determine how many users completed a transaction after interacting with a content item. The aggregate information may describe a number or grouping of online interactions (e.g., interactions with a number of content items). Additionally or alternatively, the aggregate information may describe an individual online interaction (e.g., a single interaction with a single content item). Aggregate information may include a unique identifier. In some implementations, the identifier identifies a marketing campaign. Additionally or alternatively, the identifier may uniquely identify each online interaction. In some implementations, the aggregate information describes one or more interactions associated with content items. For example, aggregate information may include a time, date, and/or location of online interactions. The interactions described by the anonymous interaction data may include viewing a content item (e.g., navigating to a webpage in which a content item is presented and/or determining that the item or a portion of the item is presented within a viewport of the device upon which the webpage is viewed, etc.), selecting/clicking a content item, hovering over a content item, and/or other interactions with a content item.

Second data processing system <NUM> may be a server, distributed processing cluster, cloud processing system, or any other computing device. Second data processing system <NUM> may include or execute at least one computer program or at least one script. In some implementations, second data processing system <NUM> includes combinations of software and hardware, such as one or more processors configured to execute one or more scripts.

Second data processing system <NUM> is shown to include database <NUM> and processing circuit <NUM>. Database <NUM> may store received data. For example, database <NUM> may store event data received from SDK <NUM> and/or supplemental data received from first data processing system <NUM>. In some implementations, database <NUM> stores identifiers. For example, database <NUM> may store event data and supplemental data sharing an intermediary identifier. The identifier may be used later for correlation of anonymous interaction data. Database <NUM> may include one or more storage mediums. The storage mediums may include but are not limited to magnetic storage, optical storage, flash storage, and/or RAM. Second data processing system <NUM> may implement or facilitate various APIs to perform database functions (i.e., managing data stored in database <NUM>). The APIs can be but are not limited to SQL, ODBC, JDBC, and/or any other data storage and manipulation API.

Processing circuit <NUM> includes processor <NUM> and memory <NUM>. Memory <NUM> may have instructions stored thereon that, when executed by processor <NUM>, cause processing circuit <NUM> to perform the various operations described herein. The operations described herein may be implemented using software, hardware, or a combination thereof. Processor <NUM> may include a microprocessor, ASIC, FPGA, etc., or combinations thereof. In many implementations, processor <NUM> may be a multi-core processor or an array of processors. Memory <NUM> may include, but is not limited to, electronic, optical, magnetic, or any other storage devices capable of providing processor <NUM> with program instructions. Memory <NUM> may include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, EEPROM, EPROM, flash memory, optical media, or any other suitable memory from which processor <NUM> can read instructions. The instructions may include code from any suitable computer programming language such as, but not limited to, C, C++, C#, Java, JavaScript, Perl, HTML, XML, Python and Visual Basic.

Memory <NUM> may include validation circuit <NUM> and analysis circuit <NUM>. Validation circuit <NUM> may validate data bundles received from SDK <NUM>. In various implementations, validation circuit <NUM> validates data bundles by verifying a signature included in the data bundles. For example, a signature may include a hash of contents of the data bundle and/or event data and validation circuit <NUM> may hash the contents of the data bundle and compare the generated hash to the received signature to determine whether the contents of the data bundle have been modified. In some implementations, if validation circuit <NUM> determines a data bundle (or specific event data included in the data bundle) to be invalid, it discards the invalid data. Additionally or alternatively, validation circuit <NUM> may flag the data as invalid which can be used by later systems to generate a confidence metric associated with the analysis results.

Analysis circuit <NUM> may receive data and produce information regarding the data. In various implementations, analysis circuit <NUM> performs statistical operations on received data to produce statistical measurements describing the received dataaa. For example, analysis circuit <NUM> may determine an interaction rate associated with a marketing campaign. In some implementations, analysis circuit <NUM> generates demographic information (e.g., user distributions, etc.), geographic results (e.g., location distributions, etc.), and/or audiences (e.g., a target group of users based on one or more parameters, for example users who purchased more than a threshold amount, etc.). In some implementations, analysis circuit <NUM> correlates event data with supplemental data. For example, analysis circuit <NUM> may correlate event data associated with an event with supplemental data associated with a content interaction using an intermediate identifier to determine an effect of the content interaction on causing the event. In various implementations, analysis circuit <NUM> generates information. The information may include an interaction rate, data describing an operation of application <NUM>, and/or the like.

Referring now to <FIG>, an improved computer architecture for securely transmitting and correlating data from mobile devices is shown, according to an illustrative implementation. In brief summary, developers may utilize reusable functions of SDK <NUM> to generate and transmit event data from client device <NUM> to first data processing system <NUM> and second data processing system <NUM> in response to events. For example, functions of SDK <NUM> embedded in a mobile application (e.g., application <NUM>, etc.) may cause client device <NUM> to collect and transmit data associated with client device <NUM> in response to a user confirming a transaction, selecting a menu option, viewing a content item, and/or the like. In response to detecting the event, SDK <NUM> may collect, bundle, and transmit data to first data processing system <NUM> and second data processing system <NUM> for analysis.

In various implementations, first data processing system <NUM> may store additional or supplemental data. For example, first data processing system <NUM> may store data associated with past customers (e.g., user preferences, etc.). As an additional example, first data processing system <NUM> may store data associated with previous user interactions with content. First data processing system <NUM> may identify supplemental data based on the data received from client device <NUM> and may transmit the supplemental data to second data processing system <NUM>. Supplemental data may include previous user interactions (e.g., a record of a user viewing a content item such as a video, a previous transaction, etc.), user demographic information, user preferences, and/or any other data. It should be understood that supplemental data may include any data that a developer has access to that is associated with a user and is not limited to the specific examples explicitly contemplated herein. In various implementations, second data processing system <NUM> correlates the data from client device and first data processing system <NUM> to determine a result. Additionally or alternatively, second data processing system <NUM> may analyze the data received from client device <NUM> to determine a result. As a non-limiting example, a user shown a video may click on the video. A content provider (e.g., first data processing system <NUM>, etc.) providing the video may wish to know how many users clicked on the video. In some implementations, users may interact with other content provided by the content provider as a result of their interaction with the first content item. For example, a user shown the video may later visit a website maintained by the content provider to purchase an item featured in the video. In some implementations, the interaction is or is associated with an online conversion. SDK <NUM> may facilitate robust reporting and analysis of user interactions with online content while preserving PII.

<FIG> illustrates a system <NUM> for transmitting and analyzing event data from mobile devices. In various implementations, client device <NUM> implements SDK <NUM> to facilitate collection and transmission of event data. For example, a developer may embed a reusable function of SDK <NUM> into application <NUM> on client device <NUM> to cause client device <NUM> to transmit data related to specific events. In various implementations, SDK <NUM> facilitates redundancy in reporting event data. For example, SDK <NUM> may enable a fallback pathway to report event data. In various implementations, application <NUM> may natively report application data to first data processing system <NUM>. SDK <NUM> may facilitate transmitting event data, redundantly and/or in parallel, to second data processing system <NUM>.

In various implementations, SDK <NUM> embedded in application <NUM> on client device <NUM> generates event data. For example, a user using application <NUM> may update a notification preference and SDK <NUM> may detect the change in preference and generate event data. The event data may include device settings (e.g., software version, hardware settings, configurations, etc.), user account settings (e.g., preferences, activity, etc.), action-sequences (e.g., clickstreams, etc.), data related to the specific event, such as the changed notification preferences, and/or the like. In some implementations, the event data describes a series of actions (e.g., clicks) the user made leading up to a purchase as well as information regarding the purchase itself (e.g., price, item purchased, etc.). In various implementations, SDK <NUM> bundles event data from a number of events. For example, SDK <NUM> may bundle event data from <NUM> events to reduce the power consumption associated with transmitting each event individually.

At step <NUM>, SDK <NUM> and/or application <NUM> may transmit the bundled event data to first data processing system <NUM>. In some implementations, the event data transmitted to first data processing system <NUM> includes identifiers. In various implementations, the event data transmitted to first data processing system <NUM> is encrypted (e.g., asymmetrically encrypted, etc.). In some implementations, SDK <NUM> may determine whether first data processing system <NUM> received the bundled event data successfully. In some implementations, if SDK <NUM> determines first data process system <NUM> did not receive the bundled event data successfully, then SDK <NUM> transmits the bundled event data to second data processing system <NUM>. Additionally or alternatively, SDK <NUM> may transmit the bundled event data to second data processing system <NUM> in parallel (step <NUM>). In various implementations, SDK <NUM> removes personal identifiers from the bundled event data before transmitting to second data processing system <NUM>. For example, SDK <NUM> may remove device identifiers and/or user identifiers before transmitting the bundled event data to second data processing system <NUM>.

In various implementations, first data processing system <NUM> identifies supplemental data based on the received bundled event data. For example, first data processing system <NUM> may identify previous interaction data associated with a user identified by a user identifier included in the bundled event data. At step <NUM>, first data processing system <NUM> may transmit the supplemental data to second data processing system <NUM>. In various implementations, first data processing system <NUM> removes personal identifiers before transmitting the supplemental data to second data processing system <NUM>. For example, first data processing system <NUM> may remove device identifiers and/or user identifiers before transmitting the supplemental data to second data processing system <NUM>.

Second data processing system <NUM> may receive the event data from client device <NUM> (e.g., via SDK <NUM>, etc.) and/or the supplemental data from first data processing system <NUM>. In various implementations, second data processing system <NUM> analyzes (e.g., performs various correlations and/or statistical operations on, etc.) the event data and/or the supplemental data to generate results. For example, second data processing system <NUM> may correlate previous interaction data included in the supplemental data with a transaction included in the event data using an intermediary identifier to determine a conversion rate.

Referring now to <FIG>, method <NUM> of reporting event data is shown, according to an implementation. In brief summary, SDK <NUM> and/or application <NUM> generates event data in response to interactions associated with a user (e.g., in-application interactions, etc.) and SDK <NUM> reports the event data to an endpoint. In various implementations, SDK <NUM> performs method <NUM>. At step <NUM>, SDK <NUM> determines a connection to first data processing system <NUM> is unavailable. For example, SDK <NUM> may ping first data processing system <NUM> to determine if it is online. In some implementations, SDK <NUM> determines whether first data processing system <NUM> confirms receipt of a data package (e.g., previous event data, etc.). In some implementations, step <NUM> is optional.

At step <NUM>, SDK <NUM> detects an event. SDK <NUM> may include various reusable functions that are embedded by developers in application <NUM>. When application <NUM> performs a particular operation, the reusable functions of SDK <NUM> may be invoked, thereby causing SDK <NUM> to perform various actions. For example, a reusable function of SDK <NUM> may be embedded in a checkout process of application <NUM> and may cause SDK <NUM> to report event data (e.g., data associated with the transaction, etc.) when the checkout is completed. Additionally or alternatively, SDK <NUM> may detect an event by listening for various indications from application <NUM>. For example, application <NUM> may generate a flag when an application event occurs and SDK <NUM> may detect the flag to identify the occurrence of an event.

At step <NUM>, SDK <NUM> may collect event data from application <NUM> and/or client device <NUM>. For example, SDK <NUM> may collect data related to a transaction associated with the event. At step <NUM>, SDK <NUM> bundles the event data. Step <NUM> may include combining event data from a number of events into a single data package. For example, event data from one-hundred events may be combined into bundled event data. In some implementations, SDK <NUM> bundles event data from events occurring over a specified time period. For example, SDK <NUM> may bundle events from each day (e.g., <NUM>-hour period, etc.). It should be understood that various bundling techniques are possible and SDK <NUM> is not limited to the specific techniques expressly enumerated herein. In various implementations, SDK <NUM> labels the event data. For example, SDK <NUM> may label the event data with an event type. The event type may correspond to a source of the event data. For example, SDK <NUM> may determine first event data corresponding to an in-application transaction is of a "transaction" type and second event data corresponding to visiting a web-page is of a "content interaction" type. In various implementations, SDK <NUM> generates different event data based on the event type. For example, SDK <NUM> may generate event data including an item price and quantity for a "transaction" event and may generate event data including a content identifier for "content interaction" type data. In should be understood that many event types are possible and are not limited to the types expressly contemplated herein.

At step <NUM>, SDK <NUM> may prepare and encrypt event data. In various implementations, step <NUM> includes removing identifiers. For example, SDK <NUM> may remove personal identifiers (e.g., device identifiers, user identifiers, etc.) from bundled event data. Additionally or alternatively, SDK <NUM> may encrypt the bundled event data. For example, SDK <NUM> may asymmetrically encrypt the bundled event data using a public key received from first data processing system <NUM>. According to the invention claimed in this European patent, SDK <NUM> removes identifiers from bundled event data that is transmitted to second data processing system <NUM>. In some implementations, SDK <NUM> encrypts bundled event data that is transmitted to first data processing system <NUM>. In some implementations, step <NUM> includes generating a signature of the bundled event data. For example, SDK <NUM> may generate a signature by hashing the bundled event data. In some implementations, step <NUM> is optional.

At step <NUM>, SDK <NUM> securely transmits event data to second data processing system <NUM>. According to the invention claimed in this European patent, SDK <NUM> transmits the event data to second data processing system <NUM> in response to determining that first data processing system <NUM> is unavailable (e.g., based on step <NUM>, etc.). According to the invention claimed in this European patent, SDK <NUM> transmits event data to second data processing system <NUM> that is free of personal identifiers, thereby preserving PII. In some implementations, SDK <NUM> transmits the bundled event data to second data processing system <NUM> in response to a threshold. For example, SDK <NUM> may transmit the bundled event data to second data processing system <NUM> in response to a threshold number of events being included in the event data bundle. In various implementations, SDK <NUM> facilitates specifying a destination (e.g., second data processing system <NUM>, etc.). For example, SDK <NUM> may facilitate receiving a URL and/or an IP address of second data processing system <NUM> specifying where to transmit event data to.

At step <NUM>, SDK <NUM> may securely transmit encrypted event data to first data processing system <NUM>. In various implementations, SDK <NUM> transmits encrypted event data including one or more identifiers to first data processing system <NUM>. In some implementations, step <NUM> is optional.

Referring now to <FIG>, method <NUM> of identifying supplemental data is shown, according to an implementation. In some implementations, first data processing system <NUM> performs method <NUM>. In some implementations, first data processing system <NUM> performs method <NUM> in response to receiving event data from client device <NUM> and/or SDK <NUM>. At step <NUM>, first data processing system <NUM> receives event data from client device <NUM>. In various implementations, first data processing system <NUM> receives event data including identifiers. For example, first data processing system <NUM> may receive event data including device identifiers. Additionally or alternatively, first data processing system <NUM> may receive encrypted event data.

At step <NUM>, first data processing system <NUM> may decrypt the event data. In various implementations, step <NUM> includes asymmetrically decrypting the event data using a private key held by first data processing system <NUM> to produce unencrypted event data. At step <NUM>, first data processing system <NUM> may select supplemental data from stored data using the event data. In various implementations, step <NUM> includes identifying supplemental data using an identifier included in the event data. In various implementations, first data processing system <NUM> searches database <NUM>. Database <NUM> may include data associated with previous content interactions. For example, first data processing system <NUM> may identify previous content interactions associated with a user. As an additional example, first data processing system <NUM> may identify a video that a user previously watched.

At step <NUM>, first data processing system <NUM> may prepare the supplemental data. For example, first data processing system <NUM> may remove personal identifiers associated with the supplemental data. In some implementations, first data processing system <NUM> removes IP addresses, user identifiers, and/or device identifiers associated with the supplemental data. Additionally or alternatively, first data processing system <NUM> may perform various obliviating functions on the supplemental data. For example, first data processing system <NUM> may fragment the supplemental data data, add noise, or perform other functions to anonymize data. At step <NUM>, first data processing system <NUM> may securely transmit the supplemental data to second data processing system <NUM>. In some implementations, first data processing system <NUM> transmits the supplemental data to second data processing system <NUM> in response to a signal. For example, second data processing system <NUM> may send a request for supplemental data to first data processing system <NUM>.

Referring now to <FIG>, method <NUM> of analyzing event data and/or supplemental data is shown, according to an implementation. In various implementations, second data processing system <NUM> performs method <NUM>. In some implementations, second data processing system <NUM> performs method <NUM> in response to receiving data from client device <NUM>. Additionally or alternatively, second data processing system <NUM> may perform method <NUM> in response to receiving a request for results from an external system (e.g., first data processing system <NUM>, etc.).

At step <NUM>, second data processing system <NUM> may receive event data from client device <NUM> (e.g., via SDK <NUM>, etc.). In various implementations, the event data is bundled event data. In various implementations, the event data does not include personal identifiers. At step <NUM>, second data processing system <NUM> may validate the received event data. In various implementations, the event data includes a signature. Second data processing system <NUM> may validate the event data using the signature. For example, second data processing system <NUM> may generate a hash of the received event data and compare the generated hash to a signature included in the event data. In some implementations, step <NUM> is optional.

At step <NUM>, second data processing system <NUM> may receive supplemental data from first data processing system <NUM>. In some implementations, second data processing system <NUM> receives supplemental data from other sources. In various implementations, the supplemental data includes interaction data associated with the event data. For example, the supplemental data may include a content interaction sharing an intermediary identifier with an event of the event data. As another example, the supplemental data may include a content item identifier that can be correlated to a transaction included in the event data. In some implementations, step <NUM> is optional. For example, in some implementations, first data processing system <NUM> does not transmit supplemental data to second data processing system <NUM>.

At step <NUM>, second data processing system <NUM> may correlate the event data to the supplemental data. For example, second data processing system <NUM> may match an identifier of a content item with an intermediary identifier of the event data to determine that the user has interacted with the content item. As an additional example, second data processing system <NUM> may determine a URL of a content item matches a website URL included in the event data. In some implementations, step <NUM> is optional. For example, second data processing system <NUM> may not receive supplemental data corresponding to every event within the event data.

At step <NUM>, second data processing system <NUM> may analyze data to generate results. In various implementations, second data processing system <NUM> analyzes event data. Additionally or alternatively, second data processing system <NUM> may analyze supplemental data. In various implementations, second data processing system <NUM> performs various analysis operations to produce the results. In various implementations, the results include a count of the number of interactions and/or event data sharing a first characteristic. Additionally or alternatively, the results may include a sum of interactions (e.g., transactions, clicks, phone calls, etc.), a sum of the value associated with each interaction (e.g., a dollar amount), and/or metadata. In various implementations, the results are aggregate information. For example, the results may include statistical information associated with event data from a number of events. In some implementations, the results are saved in database <NUM>. In various implementations, the results include data describing usage of application <NUM> in real time. For example, the results may include a count of a number of users currently viewing a specific content item. Additionally or alternatively, second data processing system <NUM> may identify a previous interaction of the user that was not performed using application <NUM> (e.g., cross-platform attribution, etc.). For example, second data processing system <NUM> may correlate a content interaction included in supplemental data to event data to determine a user previously interacted with the content on a different device than client device <NUM>.

At step <NUM>, second data processing system <NUM> may securely transmit the results to first data processing system <NUM>. Additionally or alternatively, second data processing system <NUM> may securely transmit the results to a different destination. In some implementations, step <NUM> is optional.

<FIG> illustrates a depiction of a computing system <NUM> that can be used, for example, to implement any of the illustrative systems (e.g., system <NUM>, etc.) described in the present disclosure. The computing system <NUM> includes a bus <NUM> or other communication component for communicating information and a processor <NUM> coupled to the bus <NUM> for processing information. The computing system <NUM> also includes main memory <NUM>, such as a random access memory ("RAM") or other dynamic storage device, coupled to the bus <NUM> for storing information, and instructions to be executed by the processor <NUM>. Main memory <NUM> can also be used for storing position information, temporary variables, or other intermediate information during execution of instructions by the processor <NUM>. The computing system <NUM> may further include a read only memory ("ROM") <NUM> or other static storage device coupled to the bus <NUM> for storing static information and instructions for the processor <NUM>. A storage device <NUM>, such as a solid state device, magnetic disk or optical disk, is coupled to the bus <NUM> for persistently storing information and instructions.

The computing system <NUM> may be coupled via the bus <NUM> to a display <NUM>, such as a liquid crystal display, or active matrix display, for displaying information to a user. An input device <NUM>, such as a keyboard including alphanumeric and other keys, may be coupled to the bus <NUM> for communicating information, and command selections to the processor <NUM>. In another implementation, the input device <NUM> has a touch screen display <NUM>. The input device <NUM> can include a cursor control, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor <NUM> and for controlling cursor movement on the display <NUM>.

In some implementations, the computing system <NUM> may include a communications adapter <NUM>, such as a networking adapter. Communications adapter <NUM> may be coupled to bus <NUM> and may be configured to enable communications with a computing or communications network <NUM> and/or other computing systems. In various illustrative implementations, any type of networking configuration may be achieved using communications adapter <NUM>, such as wired (e.g., via Ethernet), wireless (e.g., via WiFi, Bluetooth, etc.), pre-configured, ad-hoc, LAN, WAN, etc..

According to various implementations, the processes that effectuate illustrative implementations that are described herein can be achieved by the computing system <NUM> in response to the processor <NUM> executing an arrangement of instructions contained in main memory <NUM>. One or more processors in a multi-processing arrangement may also be employed to execute the instructions contained in main memory <NUM>. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions to implement illustrative implementations. Thus, implementations are not limited to any specific combination of hardware circuitry and software.

Although an example processing system has been described in <FIG>, implementations of the subject matter and the functional operations described in this specification can be carried out using other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

Further to the descriptions above, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs, or features described herein may enable collection of user information (e.g., information about a user's social network, social actions, or activities, profession, a user's preferences, or a user's current location), and if the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user. In situations in which the systems described herein collect personal information about users or applications installed on a user device, or make use of personal information, the users are provided with an opportunity to control whether programs or features collect user information (e.g., information about a user's social network, social actions, or activities, profession, a user's preferences, or a user's current location). In addition or in the alternative, certain data may be treated in one or more ways before it is stored or used, so that personal information is removed.

System and methods of the present disclosure offer many benefits over existing systems. Typically, data from client device applications is collected by an aggregator such as a server associated with the operation of the application before being forwarded to an analysis system for analysis. However, the connection between the client device and the aggregator is not always reliable. For example, the client device may be unable to transmit data to the aggregator and/or the aggregator may become unavailable (e.g., go offline for maintenance, etc.). The novel SDK and computer architectures described herein facilitate a redundant path for reporting event data. Specifically, the SDK of the present disclosure facilitates transmitting event data to a separate endpoint in response to a failure in transmitting data to the aggregator. Therefore, the SDK of the present disclosure improves existing systems by facilitating redundant reporting of event data and thereby reducing lost data.

Furthermore, conventional systems introduce a time delay between when client devices generate event data and when the analysis system receives the event data. As discussed earlier, typically client devices transmit event data to an aggregator that forwards the data to the analysis system. However, there may be a time delay between when the aggregator receives the data and when the analysis system receives the data. This time delay may reduce the responsiveness of an application to changes. For example, if an application has a software bug that is causing transactions to prematurely fail, a conventional system may take some time to detect the software bug. However, the novel SDK and computer architectures described herein facilitate direct transmission of event data to the analysis system. Specifically, the SDK of the present disclosure facilitates increased responsiveness of application events (e.g., crashes, bugs, etc.) by reducing the time delay between when data is generated by client devices and when an analysis system receives the data.

Moreover, generally systems that correlate data from different entities match the data using identifiers. For example, a system may match a user interaction with a user transaction using a user identifier. The novel SDK and computer architectures described herein facilitate correlation of data from different entities without revealing PII. Specifically, the SDK and computer architectures of the present disclosure facilitate correlating data from different entities using an intermediate identifier that does not reveal PII. Furthermore, content providers (e.g., application developers, etc.) may improve analytics results without having to transmit sensitive data to the analysis system. For example, the content provider may identify supplemental data from their own databases and remove sensitive identifiers (e.g., personal identifiers, etc.) before transmitting the data to the analysis system.

Implementations of the subject matter and the operations described in this specification can be carried out using digital electronic circuitry, or in computer software embodied on a tangible medium, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on one or more computer storage medium for execution by, or to control the operation of, data processing apparatus. A computer-readable storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. The computer storage medium can also be, or be included in, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices). The computer storage medium may be tangible and/or may be non-transitory.

The term "data processing apparatus" or "computing device" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example, a programmable processor, a computer, a system on a chip, or multiple ones, or combinations of the foregoing.

Circuit as utilized herein, may be implemented using hardware circuitry (e.g., FPGAs, ASICs, etc.), software (instructions stored on one or more computer readable storage media and executable by one or more processors), or any combination thereof.

Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant ("PDA"), a mobile audio or video player, a game console, a Global Positioning System ("GPS") receiver, or a portable storage device (e.g., a universal serial bus ("USB") flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

To provide for interaction with a user, implementations of the subject matter described in this specification can be carried out using a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.

Implementations of the subject matter described in this specification can be carried out using a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such backend, middleware, or frontend components.

In some implementations, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device).

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular implementations of particular inventions. Certain features that are described in this specification in the context of separate implementations can also be carried out in combination or in a single implementation. Conversely, various features that are described in the context of a single implementation can also be carried out in multiple implementations, separately, or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can, in some cases, be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination. Additionally, features described with respect to particular headings may be utilized with respect to and/or in combination with illustrative implementations described under other headings; headings, where provided, are included solely for the purpose of readability and should not be construed as limiting any features provided with respect to such headings.

Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products embodied on tangible media.

Claim 1:
A method (<NUM>) comprising:
detecting (<NUM>), by a software development kit, SDK, (<NUM>), an event and generating, by the SDK, event data in response to user interactions with a mobile application (<NUM>) on a mobile device (<NUM>), wherein the event data includes an event type;
collecting (<NUM>), by the SDK, event data for each detected event;
bundling (<NUM>), by the SDK, the event data, the method being characterized in that it further comprises :
determining (<NUM>), by the SDK, that a connection to a first data processing system (<NUM>) is unavailable;
preparing (<NUM>), by the SDK, the bundled event data for transmission to a second data processing system (<NUM>) by removing identifiers from the bundled event data; and
securely transmitting (<NUM>), by the SDK, the thus prepared bundled event data to the second data processing system (<NUM>).