Tracking provenance of digital data

A data authentication system stores a data fingerprint representing data distributed from a data source to a data recipient, allowing the data recipient to authenticate the data. The data authentication system receives, from a data source, a first data fingerprint that represents a digital entity distributed by the data source to a data recipient. A representation of the first data fingerprint is sent for storage on a blockchain. A request is received from a data recipient to authenticate the digital entity, where the request includes a second data fingerprint that represents the digital entity as distributed to the data recipient. The data authentication system authenticates the second data fingerprint against the stored first data fingerprint to verify that the data recipient received authentic data.

TECHNICAL FIELD

This disclosure relates to tracking provenance of digital data.

BACKGROUND

Digital data is commonly shared between organizations and/or individuals. As data is shared, both the provider of the data and the recipient of the data typically want to ensure that the data has not been modified in transmission. However, it can be challenging to verify that the data is unchanged, especially when the data is passed between large numbers of intermediaries or recipients or when long periods of time elapse between the creation and use of the data. Furthermore, the development of artificial intelligence is increasing the ease with which nefarious actors can tamper with data, while reducing the ability of the data's recipients to detect its illegitimacy. For example, the rise of “deep fakes,” in which voice recordings or video footage of a person are manipulated to change the person's words or actions, has made detecting data integrity an urgent issue.

DETAILED DESCRIPTION

A system and method are described for authenticating digital data. A data source generates a data fingerprint that represents digital data and stores the data fingerprint on a blockchain. Recipients of the digital data can verify the authenticity of the received data by generating a fingerprint representing the received data and comparing the generated fingerprint to the data fingerprint stored on the blockchain. Storing the fingerprint on the blockchain, rather than the data itself, preserves privacy of the data while enabling the data recipient to verify the data's provenance.

In some embodiments, a data authentication system includes a processor and a non-transitory computer readable storage medium that stores computer program instructions. When executed by the processor, the computer program instructions cause the processor to receive a data fingerprint generated by a data recipient and an identifier of data distributed to the data recipient by a data source. The data fingerprint includes an encoded representation of the data, and a representation of the data fingerprint (such as the data fingerprint itself, or a parity fingerprint that encodes multiple data fingerprints) is stored on a blockchain. Using the received identifier, the processor accesses a data fingerprint received from the data source. The data fingerprint received from the data recipient is validated against the data fingerprint received from the data source, and the data is authenticated responsive to the data fingerprint received from the data source matching the data fingerprint received from the data recipient.

In some embodiments, a method for authenticating data using data fingerprints comprises storing, by a data authentication system, a plurality of data fingerprints each received from a data source and representing data distributed by the data source to a data recipient. The data authentication system can generate a parity fingerprint encoding the plurality of data fingerprints, and write the parity fingerprint to a blockchain. A fingerprint generated by the data recipient and encoding data distributed to the data recipient by the data source can be received from the data recipient. The fingerprint received from the data recipient can be validated against the plurality of stored data fingerprints, and the stored plurality of data fingerprints can be validated using the parity fingerprint written to the blockchain.

In some embodiments, a non-transitory computer readable storage medium stores computer program instructions that, when executed by a processor, cause the processor to receive from a data source, a first data fingerprint that represents a digital entity distributed by the data source to a data recipient. The processor sends the first data fingerprint for storage on a blockchain. The processor receives a request from the data recipient to authenticate the digital entity, where the request includes a second data fingerprint that represents the digital entity as distributed to the data recipient. The processor authenticates the second data fingerprint against the stored first data fingerprint.

FIG. 1is a block diagram illustrating a system100for authenticating digital data, according to some embodiments. As shown inFIG. 1, the system100can include a data source110, a data recipient120, an authentication system130, and a blockchain135communicatively coupled over a network140. The system100can include additional or different entities than shown inFIG. 1. For example, the system100may include multiple data sources110or data recipients120, or can include other intermediate systems that transfer data between the data source110and data recipient120.

The data source110generates, maintains, or otherwise provides digital data. The data source110includes one or more computing devices capable of communicating data over the network140, such as a server, a personal computer, or a mobile phone. A person or entity may use the data source110to generate or collect data and to distribute the data to other computing devices. Alternatively, the data source110may partially or wholly autonomously aggregate data from one or more third parties before distributing the data.

The data source110can organize data into digital entities for distribution. A digital entity is a logical representation of a set of data, and any amount of data can be included in a digital entity. A digital entity can include any data type or combination of data types, including, for example, documents, photos, videos, audio files, software code, algorithms, models, graphs, and/or data sets. As one example, a digital entity is prepared by an automobile insurance company for data associated with a vehicle accident. The insurance company, which may use or communicate with the data source110, collects documents such as photos of vehicle damage, police reports, and documentation associated with the parties' insurance claims. A digital entity is formed that bundles the documentation and can be provided to the insured person, other parties to the accident, the insurance provider for the other parties, or other individuals as needed. As another example, a university prepares a digital entity that includes a student's grades for a given term. In still another example, an autonomous vehicle manufacturer packages files, such as camera frames, LiDAR telemetry, infrared sensor data, and GPS data, into a digital entity that can be distributed to vehicles. Digital entities can also be generated for sensor data collected from sensors distributed around homes, cities, or manufacturing plants. Other digital entities can be generated for medical records, experimental data, survey results, or models and datasets for artificial intelligence applications.

The data source110generates one or more data fingerprints representing a digital entity or a portion of digital data. The term “data fingerprint” is used herein to refer to an encoded representation of data, and can represent one or more of a fingerprint, a hash value, a checksum, or another encoded representation. To generate the data fingerprint, the data source110applies a function to at least a portion of the data or digital entity that outputs a representation of the data or portion of data. The function can be any of a variety of hashing functions, checksum functions, fingerprinting algorithms, or other similar functions that generate encoded representations of data. By way of example, a portion of data to which the function may be applied can include numbers or strings of text that represent important numerical values, names, or times in the data. The data source110may alternatively extract all data from the digital entity for hashing. The function is a deterministic function that, when applied to the extracted data, maps the extracted data to one or more data fingerprints. The function can be injective, mapping data values each to unique data fingerprints, or surjective, potentially mapping multiple data values to the same data fingerprint. In some cases, the function is infeasible to invert, such that the data cannot be feasibly reconstructed based on the data fingerprint alone. The same or different hash functions may be used to encode different digital entities, and the data source110may provide the hash function(s) to data recipients120when distributing the data or data entities.

The data recipient120receives a digital entity from the data source110. Like the data source110, the data recipient120can include any computing device capable of communicating data over the network140. The data recipient120can be operated by or affiliated with a person or organization who is interested in the data included in the digital entity.

The data recipient120authenticates the digital entity by generating a data fingerprint of the received entity using the same function applied by the data source110. If the data fingerprint generated by the data recipient120matches the fingerprint generated by the data source110, a user of the data recipient120can be assured that the digital entity has not been modified.

The authentication system130facilitates data authentication using the data fingerprints generated by the data source110. The authentication system130securely stores the data fingerprints from the data source110and authenticates data fingerprints received from the data recipient120against the stored fingerprints. If a data fingerprint generated by a data recipient120matches the data fingerprint generated by the data source110, the data recipient120can be assured that the data has not been changed since it was generated or compiled by the data source110.

The authentication system130interfaces between the blockchain135and the data source110or the data recipient120. The authentication system130receives and stores data fingerprints generated by the data source110for the digital entities created by the data source110. When a data recipient120requests to authenticate a digital entity, the authentication system130receives a data fingerprint generated by the data recipient and compares the received data fingerprint against the data fingerprint stored on the blockchain135.

In some cases, the authentication system130maintains a database to store data fingerprints received from the data source110. To ensure the integrity of the stored data fingerprints, the authentication system130can generate parity fingerprints that encode the contents of the database. In various embodiments, the parity fingerprints can be used in addition to the data fingerprints to authenticate data distributed to a data recipient120.

The authentication system130communicates with a blockchain135to store encoded representations of data on the blockchain135. The blockchain135includes a distributed ledger maintained by a plurality of computing devices or nodes. In various embodiments, the encoded representations of data the authentication system130writes to the blockchain135include the data fingerprints, the parity fingerprints, or both data and parity fingerprints. For example, a data fingerprint of particularly sensitive data can be written to the blockchain135, while fingerprints of less sensitive data can be stored in the database maintained by the authentication system130and only the parity fingerprint of the database written to the blockchain135.

The network140enables communication between the data source110, data recipient120, and authentication system130. The network140may include one or more local area networks (LANs), wide-area networks (WANs), metropolitan area networks (MANs), and/or the Internet.

FIG. 2is a block diagram illustrating functional modules executable by the authentication system130. As shown inFIG. 2, the authentication system130can execute an application programming interface (API)205, a parity fingerprint generator215, and a data authentication module220, as well as maintain a database210. The modules205,215,220can be software modules that are executable by a processor of the authentication system130, hardware modules, or a combination of software and hardware. Furthermore, the authentication system130can include additional, fewer, or different modules than shown inFIG. 2, and the described functionality can be distributed differently between the modules.

The API205facilitates communications between the authentication system130and one or more other computing devices, such as the data source110, data recipient120, and computing devices associated with the blockchain135. The API205can communicate with the data source110to receive data fingerprints from the data source110that represent data generated or maintained by the data source110. The API205can also write fingerprints to the blockchain135, such as the data fingerprints received from the data source110or parity fingerprints of the database210. When a data fingerprint is received from the data source110, the API205can store the fingerprint in the database210, write the fingerprint to the blockchain135, or both.

In some embodiments, the API205generates smart contracts to store data fingerprints or parity fingerprints on the blockchain135. Each smart contract stores one or more fingerprints and can be triggered by an input, such as a unique identifier of a digital entity, to output the stored fingerprint. The API205can also log requests to access the fingerprints on the blockchain. In some embodiments, the API205logs a failed access attempt on the blockchain immediately. Successful access attempts can be batched and logged on the blockchain after the batch reaches a threshold number of successful attempts or after a specified length of time.

The database210can store data fingerprints generated by the data source110. To ensure integrity of the stored fingerprints, the database210can have full global replication, audit trails, and/or point in time transaction-level backups. In various embodiments, the database210can be stored by the authentication system130(e.g., in a memory of the authentication system130) or by an external device with which the authentication system130communicates to write data to or read data from the database210.

The parity fingerprint generator215generates parity fingerprints that represent at least a portion of the database210. Each parity fingerprint can be an encoded representation of a plurality of data fingerprints in the database210, and can be generated by any of a variety of functions such as a hash function, a checksum function, or a fingerprinting algorithm. In some embodiments, the parity fingerprint generator215archives at least a portion of the database134at periodic intervals, such as once per hour, and generates a parity fingerprint of the archive. In other embodiments, the parity fingerprint generator215generates a parity fingerprint that represents a specified number of fingerprints in the database210, for example generating a parity fingerprint for each set of one hundred fingerprints added to the database210. Each parity fingerprint can be stored with an identifier of the portion of the database210that is represented by the parity fingerprint. The parity fingerprint generator215can also generate parity fingerprints on demand when a data recipient120requests authentication of data, allowing the parity fingerprint generator215to verify integrity of the data fingerprints in the database210.

The parity fingerprint generator215writes the parity fingerprints to the blockchain135or sends the parity fingerprints to the API205to write to the blockchain. Like the data fingerprints recorded on the blockchain, the parity fingerprint can be written into a smart contract that is recorded on the blockchain135and configured to output the parity fingerprint. Because the blockchain135provides immutable recordkeeping, the contents of the database210can be verified against the parity fingerprints stored on the blockchain135. The parity check can therefore provide an additional layer of trust to the database210, while reducing transactional costs and computing resources over storing the data fingerprints or the underlying data itself on the blockchain135.

The data authentication module220authenticates data distributed to the data recipient120. As described above, a data recipient120can generate a data fingerprint of data distributed to the recipient120. The data authentication module220receives the data fingerprint from the recipient120and compares the fingerprint to a data fingerprint received from the data source110. If the data fingerprint received from the recipient120matches the data fingerprint received from the data source110, the data authentication module220authenticates the data.

In some cases, when a data fingerprint received from the data source110is stored on the blockchain135, the data authentication module220authenticates the data by retrieving the data fingerprint from the blockchain135. For example, the data authentication module220sends an identifier of the data to a smart contract stored on the blockchain135, and the smart contract in response outputs the data fingerprint received from the data source110. In other cases, where the data fingerprint received from the data source110is stored in the database210, the data authentication module220authenticates the data by retrieving the data fingerprint from the database210and retrieving the parity fingerprint for the portion of the database210that includes the data fingerprint from the blockchain135. The data authentication module220can also generate, or request from the parity fingerprint generator215, a parity fingerprint indicating the state of the database210when the data fingerprint was retrieved. The generated parity fingerprint, indicating the state of the database at the time of the fingerprint retrieval, can be compared to the parity fingerprint stored on the blockchain to authenticate the data fingerprints in the database210. If there is a match between both the data fingerprint received from the data recipient120and the data fingerprint stored in the database210, as well as the parity fingerprint indicating the current state of the database and the parity fingerprint stored on the blockchain135, the data authentication module220authenticates the data that was distributed to the data recipient120.

FIG. 3is an interaction diagram illustrating a process300for authenticating digital data, according to some embodiments. As shown inFIG. 3, the process300can include interactions between the data source110, the blockchain135, and the data recipient120. The process300can include additional, fewer, or different steps, and the steps can be performed in different orders.

As shown inFIG. 3, the data source110can assemble306data into a digital entity. The digital entity may represent a single data item (e.g., a number), a collection of data items (e.g., numbers extracted from a spreadsheet), a collection of one or more files (e.g., one or more documents each stored as a unique file), a code repository, or any other data structure, file structure, or logical representation of a set of digital data. The data source110generates308a data fingerprint for at least a portion of the digital entity. Alternatively, the data source110can generate308the data fingerprint for data that is not part of a digital entity.

The data source110records the data fingerprint on the blockchain125, optionally using the API205. The data source110sends310the data fingerprint to the API205, which records312the data fingerprint of the digital entity on the blockchain135. When a data fingerprint135is recorded on the blockchain via the API205, the API outputs314an identifier of the entity and a timestamp indicating when the data fingerprint was recorded. The entity identifier and the timestamp may be output within a token generated by the API, such as a JSON web token.

In one embodiment, the data fingerprint is recorded to the blockchain135via a smart contract. The authentication system API205writes the data fingerprint into the smart contract, which defines a protocol by which a user or system, such as the data recipient120, can access the data fingerprint value. For example, the smart contract comprises computer program code that is configured to cause the smart contract to output the data fingerprint value in response to receiving an identifier of the digital entity corresponding to the contract.

After the data fingerprint is recorded on the blockchain135, the data source110can share the data with any desired target audience(s) affiliated with one or more of the data recipients120. To share the data, the data source110can distribute316the digital entity and the identifier of the entity and timestamp output by the authentication system API205. The data source110may use any of a variety of communication channels to send the digital entity to the data recipient120, including both electronic channels (such as transmitting data over the Internet) and physical channels (such as mailing a physical package to a person associated with the data recipient120).

The data recipient120can authenticate the entity at any point after receiving it by generating318a data fingerprint of the received entity. The data fingerprint of the received entity is generated using the same fingerprint function used by the data source110. The data recipient120sends320the generated data fingerprint and the entity identifier and timestamp received from the data source110to the authentication system API205.

The authentication system API205triggers322the smart contract associated with the entity identifier. When triggered, the smart contract outputs the data fingerprint stored on the blockchain by the data source110. If the output fingerprint matches the data fingerprint received from the data recipient120, the API205returns324a validation result indicating that the data entity is valid. If the fingerprints do not match, the validation result returned by the API205indicates that the data entity has been modified.

FIG. 4is an interaction diagram illustrating another process400for authenticating digital data using a blockchain-backed parity check, according to some embodiments. As shown inFIG. 4, the process400comprises interactions between the data source110, the authentication system130, and the data recipient120. The process400can include additional, fewer, or different steps, and the steps can be performed in different orders.

As shown inFIG. 4, the data source110generates402a digital fingerprint for digital data. The data fingerprinted by the data source110may include a single data item (e.g., a number), a collection of data items (e.g., numbers extracted from a spreadsheet), a collection of one or more files (e.g., one or more documents each stored as a unique file), a code repository, or any other data structure, file structure, or logical representation of a set of digital data. At least a portion of the data is represented by the digital fingerprint. The data source110may be triggered to generate402the fingerprint when the data is created or modified, when the data is requested by a data recipient120, or at a preset time.

The data source110sends404the fingerprint to the authentication system130, which stores406the fingerprint in the database134. The authentication system130generates a data identifier uniquely identifying the fingerprint and returns408the data ID to the data source110. The data source110retains the data ID to map the data to the fingerprint stored in the database134.

At periodic intervals, the authentication system130archives at least a portion of the database134and generates410a parity fingerprint of the archived portion. The parity fingerprint is an encoded representation of the fingerprints stored in the database134, and may be generated, for example, approximately once per hour. The authentication system130writes412each parity fingerprint to a blockchain135.

After the data fingerprint has been recorded in the database134, the data source110can share the data with any desired target audience(s) affiliated with one or more of the data recipients120. To share the data, the data source110distributes414the data and the data identifier output by the authentication system130. The data source110may use any of a variety of communication channels to send the data to the data recipient120, including both electronic channels (such as transmitting data over the Internet) and physical channels (such as mailing a physical package to a person associated with the data recipient120).

The data recipient120can authenticate the data at any point after receiving it by generating416a digital fingerprint of the received data. The fingerprint of the received data is generated using the same fingerprinting function used by the data source110. The data recipient120sends418the generated fingerprint and the data identifier to the authentication system130for validation.

The authentication system130accesses the fingerprint stored in the database134using the data identifier and validates420the received fingerprint against the stored fingerprint. In some cases, either automatically or at the request of a data recipient120, the authentication system130may also validate422the fingerprint stored in the database134using the parity fingerprint stored on the blockchain135. For example, the authentication system130calculates a parity fingerprint of at least a portion of the database134including the fingerprint to be validated. The calculated parity fingerprint is compared to the parity fingerprint stored on the blockchain135to verify that the fingerprints match. If the fingerprints match, the authentication system130determines the fingerprint stored in the database134to be correct.

If the fingerprint stored in the database134matches the fingerprint received from the data recipient120, the authentication system130returns424a validation result indicating that the data entity is valid. The validation result may also verify that the stored fingerprint has been authenticated against the parity fingerprint on the blockchain135. If the fingerprints do not match, the validation result returned by the authentication system130indicates that the data entity has been modified.

According to the example processes described with respect toFIGS. 3-4, the data recipient120can use the validation result to verify the accuracy of data. The data recipient120can authenticate data using the process described with respect toFIG. 2any number of times, and any length of time after the data was created. For example, a data recipient120can verify the integrity of data months or years after a data source110generated or collected the data, and after the data has passed through one or more intermediaries between the data source110and the data recipient120.

In an example use case of the process described inFIGS. 3-4, the data source110is a software supplier for autonomous vehicles and the data recipient120is an autonomous vehicle. When the software supplier updates code for the vehicle, the supplier registers the updated code by generating a fingerprint representing the code and storing the fingerprint on the blockchain135. Once it receives the update from the software supplier, the autonomous vehicle can verify that it is executing the correct code by similarly generating a fingerprint and authenticating the generated fingerprint against the stored fingerprint. The autonomous vehicle can verify software integrity and data currency at any time using the process described inFIGS. 3-4. For example, the autonomous vehicle can verify its software and GPS data before each trip.

FIG. 5is a block diagram illustrating an example of a processing system500in which at least some operations described herein can be implemented. For example, one or more of the data source110, data recipient120, or authentication system130may be implemented as the example processing system500. The processing system500may include one or more central processing units (“processors”)502, main memory506, non-volatile memory510, network adapter512(e.g., network interfaces), video display518, input/output devices520, control device522(e.g., keyboard and pointing devices), drive unit524including a storage medium526, and signal generation device530that are communicatively connected to a bus516. The bus516is illustrated as an abstraction that represents any one or more separate physical buses, point to point connections, or both connected by appropriate bridges, adapters, or controllers. The bus516, therefore, can include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 594 bus, also called “Firewire.”

In various embodiments, the processing system500operates as part of a user device, although the processing system500may also be connected (e.g., wired or wirelessly) to the user device. In a networked deployment, the processing system500may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

The processing system500may be a server computer, a client computer, a personal computer, a tablet, a laptop computer, a personal digital assistant (PDA), a cellular phone, a processor, a web appliance, a network router, switch or bridge, a console, a hand-held console, a gaming device, a music player, network-connected (“smart”) televisions, television-connected devices, or any portable device or machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the processing system500.

While the main memory506, non-volatile memory510, and storage medium526(also called a “machine-readable medium) are shown to be a single medium, the term “machine-readable medium” and “storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store one or more sets of instructions528. The term “machine-readable medium” and “storage medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the computing system and that cause the computing system to perform any one or more of the methodologies of the presently disclosed embodiments.

Moreover, while embodiments have been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments are capable of being distributed as a program product in a variety of forms, and that the disclosure applies equally regardless of the particular type of machine or computer-readable media used to actually effect the distribution. For example, the technology described herein could be implemented using virtual machines or cloud computing services.

The network adapter512enables the processing system500to mediate data in a network514with an entity that is external to the processing system500through any known and/or convenient communications protocol supported by the processing system500and the external entity. The network adapter512can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.

As indicated above, the techniques introduced here implemented by, for example, programmable circuitry (e.g., one or more microprocessors), programmed with software and/or firmware, entirely in special-purpose hardwired (i.e., non-programmable) circuitry, or in a combination or such forms. Special-purpose circuitry can be in the form of, for example, one or more application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc.