Methods, systems, and computer readable media for calibration testing and traceability using a distributed ledger

A method that occurs at a calibration system includes generating a first digital record based on a calibration of the calibration system using a calibration source; providing a copy of the first digital record to a first device under test (DUT); and providing a calibration test signal associated with a calibration of the first DUT, wherein the first DUT uses the first digital record in generating a second digital record based on the calibration of the first DUT.

TECHNICAL FIELD

The subject matter described herein relates to distributed ledger applications. More specifically, the subject matter relates to methods, systems, and computer readable media for calibration testing and traceability using a distributed ledger.

BACKGROUND

Manufacturers generally test and calibrate equipment for various purposes, e.g., government compliance and quality assurance. For example, a manufacturer may utilize calibration devices or labs to test that one or more pieces of equipment are performing as expected and meet particular accuracy, with traceability to National Metrology Institute (NMI) standards. In this example, the one or more pieces of equipment may also be calibrated or adjusted to improve performance or meet standards. After calibration is performed, calibration information (e.g., certificates, devices involved in the calibration, calibration settings, calibration operator details, location of calibration, traceability documents, and/or other related details) may be generated and stored for future audits or inspections by the manufacturer and/or other parties. As the amount of calibration information grows, the calibration information may become increasing difficult to store in a scalable and orderly manner. Further, the retrieval of the calibration information may require humans to identify, approve, and provide various documents, which can be time consuming and labor intensive. Furthermore, authenticity of the calibration information can be difficult to validate, e.g., since documents may be forgeable and/or may lack proof of authenticity, e.g., from a trusted source.

Accordingly, a need exists for methods, systems, and computer readable media for calibration testing and traceability using a distributed ledger.

SUMMARY

Methods, systems, and computer readable media for calibration testing and traceability using a distributed ledger are disclosed. One method occurs at a calibration system implemented using at least one processor. The method includes generating a first digital record based on a calibration of the calibration system using a calibration source; providing a copy of the first digital record to a first device under test (DUT); and providing a calibration test signal associated with a calibration of the first DUT, wherein the first DUT uses the first digital record in generating a second digital record based on the calibration of the first DUT.

A system includes a distributed ledger system implemented using at least one processor. The distributed ledger system is configured for: generating a first digital record based on a calibration of the calibration system using a calibration source; providing a copy of the first digital record to a first device under test (DUT); and providing a calibration test signal associated with a calibration of the first DUT, wherein the first DUT uses the first digital record in generating a second digital record based on the calibration of the first DUT.

As used herein, the term ‘node’ refers to at least one physical computing platform including one or more processors, network interfaces, and memory.

As used herein, each of the terms ‘engine’ and ‘module’ refers to hardware, which may also include software and/or firmware, for implementing the feature(s) being described.

DETAILED DESCRIPTION

The subject matter described herein relates to methods, systems, and computer readable media for calibration testing and traceability using a distributed ledger. When calibrating physical or virtual computing equipment or components thereof, it may be desirable to store calibration information (e.g., certifications, reports, calibration results, details about the calibration, etc.) in a secure format. For example, a party, e.g., an auditor or a manufacturer, may want to verify and check traceability of calibration information for equipment or components therein. In this example, if calibration information is stored in various formats (e.g., paper documents and digital files) and/or in multiple filing systems, the retrieval of the calibration information can be time consuming and resource intensive, especially if a human is required. Further, the authenticity and accuracy of the calibration information can be difficult to validate, especially if the calibration information is stored in physical files (e.g., paper documents) or insecure digital files.

In accordance with some aspects of the subject matter described herein, techniques, methods, systems, and/or mechanisms may use one or more distributed ledgers (e.g., blockchains or Merkle tree data structures) to store one or more digital records (e.g., digital certificates and/or hash values) associated with calibration information or related data. For example, a distributed ledger in accordance with some aspects described herein may include blocks or portions containing digital records associated with calibration information for a unit (e.g., a calibrated device, a system of multiple devices, or a calibration system). In this example, a calibration system may generate a first digital record when the calibration system is calibrated using a calibration source and a device under test that is calibrated using the calibration system may generate a second digital record that is based at least in part on the first digital record. Continuing with this example, since digital records may be based on related digital records, since blocks of the distributed ledger may be cryptographically linked (e.g., modifications of a block would require modifications to subsequent blocks), and since the calibration information may be encrypted in an external data store, the data integrity of the calibration information is assured, and the calibration information is secure from unauthorized changes.

In accordance with some aspects of the subject matter described herein, techniques, methods, systems, and/or mechanisms may generate and store system-level digital records (e.g., digital certificates and/or hash values) associated with calibration of multiple components or elements of a system. For example, in an autonomous vehicle, a sensor controller may receive digital certificates containing calibration information for various sensors in the autonomous vehicle. In this example, the sensor controller may generate a system-level digital certificate (e.g., a hash value generated using a SHA-3 function) based on at least the digital certificates associated with the various devices in the autonomous vehicle. In some embodiments, the system-level digital certificate may be stored in a distributed ledger and/or one or more data stores (e.g., databases or data storage systems).

FIG. 1is a diagram illustrating an example computing environment100for a distributed ledger enabled calibration system. Referring toFIG. 1, computing environment100may include CS102and one or more device(s) and/or system(s) under test (DUT)116. CS102may represent any suitable entity or entities (e.g., one or more computing platforms, nodes, or devices) associated with testing or calibrating DUT116(e.g., manufacturing equipment). For example, CS102may generate and send traffic to DUT116and/or receive traffic from DUT116and may analyze one or more aspects associated with DUT116.

In some embodiments, CS102may include a stand-alone tool, a testing device, or software executing on one or more processor(s). In some embodiments, CS102may be a single device or node or may be distributed across multiple devices or nodes. In some embodiments, CS102may include one or more modules for performing various test related functions. For example, CS102may include a signal generator for generating test waveforms, sending the test waveforms to DUT116, receiving waveforms from DUT116, and analyzing performance and functionality of DUT116.

DUT116may be any suitable entity or entities (e.g., devices, systems, or platforms) for receiving, processing, forwarding, and/or sending one or more messages. In some embodiments, DUT116may include network equipment, industrial equipment, or one or more devices that are being tested and/or calibrated, e.g., to a national standard, by CS102. For example, DUT116may include a security device (e.g., firewall or an intrusion protection system (IPS)) that inspects traffic that traverses the security device (e.g., Internet protocol (IP) packets and/or network communications). In this example, a compliance or certification test may be performed to determine whether DUT116meets a related compliance or certification standard.

CS102may include a test controller (TC)104, a distributed ledger manager (DLM)106, data storage108, one or more processors110, and memory112. TC104may be any suitable entity or entities (e.g., software executing on a processor, a field-programmable gateway array (FPGA), and/or an application-specific integrated circuit (ASIC), or a combination of software, an FPGA, and/or an ASIC) for performing one or more aspects associated with testing or calibrating DUT116and/or various aspects thereof.

In some embodiments, TC104may be implemented using one or more processors110and/or memory112. For example, TC104may utilize one or more processors110(e.g., executing software stored in memory112) to generate test packets for a number of calibration routines or tests (e.g., flows or sessions). In another example, TC104may also utilize one or more of processors110to perform or initiate analyses involving test waveforms and/or related responses from DUT116.

In some embodiments, TC104may include or provide at least one communications interface for communicating with an operator114(e.g., a management system or a human operator). For example, operator114may be any entity (e.g., an automated system or a device or system controlled or controllable by a human user) for selecting and/or configuring various aspects associated with calibrating and/or generating or configuring calibration settings. In some embodiments, various user interfaces, e.g., an application programming interface (API) and a graphical user interface (GUI), may be available so that operator114can provide CS102or TC104with configuration information (e.g., tests to be performed, types of metrics or statistics to be generated and/or measured, and/or other settings) and/or for controlling (e.g., pause, restart, or stop) a test or calibration session.

DLM106may be any suitable entity or entities (e.g., software executing on a processor, an ASIC, an FPGA, or a combination of software, an ASIC, and/or an FPGA) for performing one or more actions associated with storing, accessing, and managing information in one or more distributed ledgers or related data stores (e.g., data storage108). For example, DLM106may receive calibration information or other related data may generate digital signatures (e.g., integrity hash values) based on the calibration information. In this example, DLM106may, e.g., using a blockchain client, add or store the calibration information and/or related digital signatures in a distributed ledger.

In some embodiments, DLM106may communicate with CS102and/or other related entities (e.g., TC104) to receive, process, or send calibration information and/or related data, e.g., an access security key to allow an entity to access and/or decrypt calibration information. For example, DLM106may receive calibration information or other related data from CS102or TC104, may generate a hash value for calibration information, may store the hash value in a blockchain for verifying the calibration information (e.g., when the calibration information is obtained from a data store at a later date by a third party), may also encrypt the calibration information using one or more encryption keys and store the calibration information in one or more data stores, e.g., data storage108. In this example, DLM106may also receive information requests from various entities and may provide access security keys and/or decrypted calibration information.

In some embodiments, TC104and/or DLM106may include functionality for accessing data storage108or other memory. Data storage108may be any suitable entity or entities (e.g., a storage device, memory, a non-transitory computer readable medium, or a storage system) for maintaining or storing information related to testing and/or calibration. In some embodiments, data storage108and/or memory may be located at CS102, another node, or distributed across multiple platforms or devices.

In some embodiments, data storage108may include one or more data stores for storing different types of calibration information. For example, some unencrypted data may be stored in a first data store, and other encrypted data may be stored in one or more data stores separate from the first data store.

In some embodiments, data storage108may contain calibration information, also referred to herein as traceability information, usable for auditing or verifying traceability of a calibrated device. Example calibration information related to a calibration event or related device may include certification information, calibration settings, calibration results, devices involved in the calibration, measurement reports, parties involved with the calibration, calibration location, and/or other details associated with the calibration.

In some embodiments, CS102, DLM106, and/or another entity may generate a first digital record (e.g., a hash-based digital certificate) based on a calibration of CS102using a calibration source; may provide a copy of the first digital record to a DUT116; and may provide a calibration test signal associated with a calibration of DUT116, where DUT116uses the first digital record in generating a second digital record based on the calibration of DUT116.

It will be appreciated thatFIG. 1is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 1may be changed, altered, added, or removed.

FIG. 2is a diagram illustrating calibration of CS102using a calibration source202and generation of a digital record (e.g., a digital certificate204or a related hash value) related to the calibration of CS102. In some embodiments, calibration source202may represent a National Metrology Institute (NMI), e.g., the National Physical Library (NPL) of the United Kingdom or the National Institute of Standards and Technology (NIST) of the United States.

Referring toFIG. 2, CS102may execute one or more predefined test routines for calibrating CS102. For example, CS102may execute a calibration test routine involving calibration source202. In this example, in response to the calibration test routine (e.g., during and/or after completing the calibration test routine), CS102may adjust or modify one or more calibration parameter values (e.g., device settings) within CS102. Continuing with this example, post-calibration parameter values may be usable for calibrating CS102such that CS102complies with one or more standards associated with the calibration test routine.

In some embodiments, after performing calibration or a portion thereof, CS102or another entity may generate a digital certificate204indicating various details about the performed calibration or the portion thereof. In some embodiments, digital certificate204may represent any digital record containing details (e.g., administrative data, calibration settings, calibration site, etc.) about a performed calibration or portion thereof. In some embodiments, digital certificate204and/or a hash value therein may be generated using one or more hash functions and one or more inputs. For example, to generate digital certificate204and/or a hash value therein, a hash function may use one or more criteria as input.

In some embodiments, input criteria for a hash function (e.g., a SHA-1, SHA-2, or SHA-3 function) that generates digital certificate204and/or a related hash value may include post-calibration parameter values, calibration source identification information, calibration system identification information, calibration routine information, and/or timestamp information. For example, using various data as input, CS102may use a SHA-256 function to produce 256-bit hash value that is associated with the calibration of CS102(e.g., completion of one or more predefined calibration test routines involving CS102). In another example, using various data as input, CS102may use a SHA-512 function to produce 512-bit hash value that is associated with the calibration of CS102(e.g., completion of one or more predefined calibration test routines involving CS102).

It will be appreciated thatFIG. 2is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 2may be changed, altered, added, or removed.

FIG. 3is a diagram illustrating storing a digital record (e.g., digital certificate300or a related hash value) related to the calibration of the DUT in distributed ledger302. Referring toFIG. 3, after CS102generates digital certificate204, CS102or DLM106may store or facilitate storing digital certificate300(e.g., a copy of digital certificate204or an encrypted version thereof) in distributed ledger302. In some embodiments, digital certificate204, a version thereof (e.g., digital certificate300), and/or other information may be stored in storage local to CS102, e.g., data storage108and/or may be stored in external storage accessible to CS102or DLM106. For example, secure external storage is accomplished via a distributed ledger or blockchain. In this example, DLM106may be configured to provide blockchain client functionality such that DLM106can post digital certificate600as a blockchain transaction to distributed ledger302. In some embodiments, DLM106or other related entities may perform various actions to facilitate adding the blockchain transaction to the blockchain.

It will be appreciated thatFIG. 3is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 3may be changed, altered, added, or removed.

FIG. 4is a diagram illustrating sending a copy of the digital record related to the calibration of CS102from CS102to DUT116. Referring toFIG. 4, prior to or concurrently with calibration of DUT116, CS102may send a digital certificate400(e.g., a copy of digital certificate204or an encrypted version thereof) to DUT116. For example, prior to CS102completing a calibration test involving DUT116(e.g., an electronic sensor in an autonomous vehicle), a copy of the digital certificate hash value is communicated to DUT116.

In some embodiments, DUT116may store digital certificate400and, when calibration of DUT116is completed, DUT116may generate, using digital certificate400and other data, a digital certificate related to the calibration of DUT116

It will be appreciated thatFIG. 4is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 4may be changed, altered, added, or removed.

FIG. 5is a diagram illustrating calibration of DUT116and generation of a digital record (e.g., a digital certificate500or a related hash value) related to the calibration of DUT116. Referring toFIG. 5, DUT116may execute one or more predefined calibration test routines. For example, CS102may facilitate a calibration test routine involving DUT116by sending test signals or responding to DUT116during the execution of the calibration test routine. In this example, in response to the calibration test routine (e.g., during and/or after completing the calibration test routine), DUT116may adjust or modify one or more calibration parameter values (e.g., device settings) within DUT116. Continuing with this example, post-calibration parameter values may be usable for calibrating DUT116such that DUT116complies with one or more standards associated with the calibration test routine.

In some embodiments, after performing calibration or a portion thereof, DUT116or another entity may generate a digital certificate500indicating various details about the performed calibration or the portion thereof. In some embodiments, digital certificate500may represent any digital record containing details about a performed calibration or portion thereof. In some embodiments, digital certificate500and/or a hash value therein may be generated using one or more hash functions and one or more inputs. For example, to generate digital certificate500and/or a hash value therein, a hash function may use one or more criteria as input.

In some embodiments, input criteria for a hash function (e.g., a SHA-1, SHA-2, or SHA-3 function) that generates digital certificate500and/or a related hash value may include digital certificate400or related data from CS102, post-calibration parameter values, calibration source identification information, calibration system identification information, calibration routine information, and/or timestamp information. For example, using digital certificate400and various other data as input, DUT116may use a SHA-256 function to produce 256-bit hash value that is associated with the calibration of DUT116(e.g., completion of one or more predefined calibration test routines involving DUT116).

It will be appreciated thatFIG. 5is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 5may be changed, altered, added, or removed.

FIG. 6is a diagram illustrating storing a digital record (e.g., digital certificate600or a related hash value) related to the calibration of the DUT in distributed ledger302. Referring toFIG. 6, after DUT116generates digital certificate500, DUT116or DLM106may store or facilitate storing digital certificate600(e.g., a copy of digital certificate500or an encrypted version thereof) in distributed ledger302. In some embodiments, digital certificate500, a version thereof (e.g., digital certificate600), and/or other information may be stored in storage local to DUT116, e.g., data storage502and/or may be stored in external storage accessible to DUT116or DLM106. For example, secure external storage is accomplished via a distributed ledger or blockchain. In this example, DLM106may be configured to provide blockchain client functionality such that DLM106can post digital certificate600as a blockchain transaction to distributed ledger302. In some embodiments, DLM106or other related entities may perform various actions to facilitate adding the blockchain transaction to the blockchain.

It will be appreciated thatFIG. 6is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 6may be changed, altered, added, or removed.

FIG. 7is a diagram illustrating an example system environment700for generating a system-level hash value. System environment700may represent various networks or related devices or systems, e.g., an IoT device or network, an autonomous vehicle or system therein (e.g., a sensor system), or another network or system. Referring toFIG. 7, system environment700may include a system controller702, a distributed ledger704, and a plurality of DUTs708A-C representing one or more device, components, or equipment in system environment700.

System controller702may represent any suitable entity or entities (e.g., one or more computing platforms, nodes, or devices) associated with generating a system-level hash value or a related digital certificate714. In some embodiments, system controller702may include or utilize one or more modules, processors, communications interfaces, and/or memory, e.g., data storage706.

In some embodiments, system controller702may receive information and/or digital certificates710A-C from various components or nodes within system environment700and using that information may determine or generate information about the state of system environment700. For example, system controller702may include one or more communications interfaces for receiving digital certificates710A-C or other information from DUTs708A-C.

In some embodiments, system controller702or a related entity may generate a system-level hash value and/or certificate712. In such embodiments, the system-level hash value and/or certificate712may be based on a combination of some or all of digital certificates710A-C received from DUTs708A-C. For example, system controller702or a related entity generate certificate712using a hash function (e.g., a SHA-1, SHA-2, or SHA-3 function) that uses hash values or information from digital certificates710A-C as input.

In some embodiments, system controller702may include data storage706for storing digital certificates710A-C, digital certificate712, and/or other information. For example, after receiving digital certificates710A-C, system controller702may store these digital certificates710A-C and/or related information in data storage706. In this example, after using digital certificates710A-C to generate digital certificate714, system controller702may store digital certificate712in data storage706.

In some embodiments, system controller702may include a DLM106for interacting with a distributed ledger704, e.g., accessing data, storing data, or deleting data therein. For example, after generating a digital certificate712, DLM106may generate a transaction for adding digital certificate714(e.g., a copy of digital certificate714or an encrypted version thereof) to distributed ledger704.

In some embodiments, system controller702may be a sensor controller, e.g., associated with an autonomous vehicle, industrial equipment, an IoT system, or other device. In such embodiments, each of DUTs708A-C may be a sensor (e.g., an autonomous vehicle sensor, an avionics sensor, an internet of thing (IoT) sensor, etc.) associated with system environment700.

It will be appreciated thatFIG. 7is for illustrative purposes and that various depicted entities, their locations, and/or their functions described above in relation toFIG. 7may be changed, altered, added, or removed.

FIG. 8is a diagram illustrating an example process800for calibration testing and traceability using a distributed ledger. In some embodiments, process800, or portions thereof, may be performed by or at CS102, DLM106, system controller702, and/or another node or module. In some embodiments, process800may include steps802,804,806, and/or806.

Referring to process800, in step802, a first digital record may be generated based on a calibration of the calibration system using a calibration source. For example, after calibrating CS102using a reference signal from calibration source202, CS102or a related entity may generate digital certificate204and store digital certificate204in data storage108.

In step802, a copy of the first digital record may be provided to a first DUT. For example, CS102or a related entity may send digital certificate400(e.g., a copy of digital certificate204) to DUT116.

In step804, a calibration test signal associated with a calibration of the first DUT may be provided to the first DUT. In some embodiments, the first DUT uses the first digital record in generating a second digital record based on the calibration of the first DUT. For example, CS102may send a calibration test signal when DUT116is executing a calibration test routine for calibrating DUT116. In this example, after calibrating DUT116using calibration test routine, DUT116or a related entity may generate digital certificate500, where digital certificate500is generated using digital certificate400or information therein.

In some embodiments, the first digital record may include a first hash value, wherein the first hash value may be generated using a first hash function (e.g., a SHA-256 function) that uses first calibration parameter values as input, wherein the first calibration parameter values may be based on the calibration of the calibration system.

In some embodiments, the first hash function uses calibration source identification information, calibration system identification information, calibration routine information, and/or timestamp information as input.

In some embodiments, the second digital record may include a second hash value, wherein the second hash value may be generated using a second hash function (e.g., a SHA-256 function) that uses second calibration parameter values and the first digital record as input, wherein the second calibration parameter values may be determined based on the calibration of the first DUT.

In some embodiments, the first digital record may be stored in a distributed ledger, e.g., a blockchain.

In some embodiments, the second digital record may be stored in a distributed ledger by the first DUT, the calibration system, or another entity.

In some embodiments, the first DUT may include a sensor, an autonomous vehicle sensor, an avionics sensor, a network device, an internet of thing (IoT) device, or software.

In some embodiments, a calibration system or a system controller may be configured for receiving a plurality of digital records associated with calibrations of a plurality of DUTs in a related system (e.g., an autonomous vehicle, a computer network, or an IoT network), wherein the second digital record may be one of the plurality of digital records and wherein the first DUT may be one of the plurality of DUTs in the related system; generating, using the plurality of digital records, a system-level hash value that may be indicative of a calibration state of the related system; and storing the system-level hash value or a related digital record in a distributed ledger.

In some embodiments, a system-level hash value or a related digital record may be generated using a hash function that generates a same or larger number of bits than individual DUTs. For example, a system-level hash value may be a 512-bit hash value and a DUT hash value may be a 256-bit hash value.

It will be appreciated that process800is for illustrative purposes and that different and/or additional actions may be used. It will also be appreciated that various actions described herein may occur in a different order or sequence.

It should be noted that CS102, DLM106, system controller702, and/or functionality described herein may constitute a special purpose computing device. Further, CS102, DLM106, system controller702, and/or functionality described herein can improve the technological fields of secure data storage, calibration testing, calibration traceability, and/or calibration auditing. For example, by using one or more distributed ledgers (e.g., one or more blockchains) and using hash-based digital certificates, calibration information may be stored in a secured and verifiable accurate format.