Package: azure-confidentialledger
Azure Confidential Ledger client library for PythonAzure Confidential Ledger provides a service for logging to an immutable, tamper-proof ledger. As part of theAzure Confidential Computingportfolio, Azure Confidential Ledger runs in secure, hardware-based trusted execution environments, also known as enclaves. It is built on Microsoft Research'sConfidential Consortium Framework.Source code|Package (PyPI)|Package (Conda)|API reference documentation|Product documentationGetting startedInstall packagesInstallazure-confidentialledgerandazure-identitywithpip:pipinstallazure-identityazure-confidentialledgerazure-identityis used for Azure Active Directory
authentication as demonstrated below.PrerequisitesAnAzure subscriptionPython 3.6 or laterA running instance of Azure Confidential Ledger.A registered user in the Confidential Ledger, typically assigned duringARMresource creation, withAdministratorprivileges.Authenticate the clientUsing Azure Active DirectoryThis document demonstrates usingDefaultAzureCredentialto authenticate to the Confidential Ledger via Azure Active Directory. However,ConfidentialLedgerClientaccepts anyazure-identitycredential. See theazure-identitydocumentation for more information about other credentials.Using a client certificateAs an alternative to Azure Active Directory, clients may choose to use a client certificate to authenticate via mutual TLS.azure.confidentialledger.ConfidentialLedgerCertificateCredentialmay be used for this purpose.Create a clientDefaultAzureCredentialwill automatically handle most Azure SDK client scenarios. To get started, set environment variables for the AAD identity registered with your Confidential Ledger.exportAZURE_CLIENT_ID="generated app id"exportAZURE_CLIENT_SECRET="random password"exportAZURE_TENANT_ID="tenant id"Then,DefaultAzureCredentialwill be able to authenticate theConfidentialLedgerClient.Constructing the client also requires your Confidential Ledger's URL and id, which you can get from the Azure CLI or the Azure Portal. When you have retrieved those values, please replace instances of"my-ledger-id"and"https://my-ledger-id.confidential-ledger.azure.com"in the examples below. You may also need to replace"https://identity.confidential-ledger.core.azure.com"with the hostname from theidentityServiceUriin the ARM description of your ledger.Because Confidential Ledgers use self-signed certificates securely generated and stored in an enclave, the signing certificate for each Confidential Ledger must first be retrieved from the Confidential Ledger Identity Service.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)Conveniently, theConfidentialLedgerClientconstructor will fetch the ledger TLS certificate (and write it to the specified file) if it is provided with a non-existent file. The user is responsible for removing the created file as needed.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.identityimportDefaultAzureCredentialcredential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path="ledger_certificate.pem")# The ledger TLS certificate is written to `ledger_certificate.pem`.To make it clear that a file is being used for the ledger TLS certificate, subsequent examples will explicitly write the ledger TLS certificate to a file.Key conceptsLedger entries and transactionsEvery write to Azure Confidential Ledger generates an immutable ledger entry in the service. Writes, also referred to as transactions, are uniquely identified by transaction ids that increment with each write. Once written, ledger entries may be retrieved at any time.CollectionsWhile most use cases involve just one collection per Confidential Ledger, we provide the collection id feature in case semantically or logically different groups of data need to be stored in the same Confidential Ledger.Ledger entries are retrieved by theircollectionId. The Confidential Ledger will always assume a constant, service-determinedcollectionIdfor entries written without acollectionIdspecified.UsersUsers are managed directly with the Confidential Ledger instead of through Azure. Users may be AAD-based, identified by their AAD object id, or certificate-based, identified by their PEM certificate fingerprint.ReceiptsTo enforce transaction integrity guarantees, an Azure Confidential Ledger uses a [Merkle tree][merkle_tree_wiki] data structure to record the hash of all transactions blocks that are appended to the immutable ledger. After a write transaction is committed, Azure Confidential Ledger users can get a cryptographic Merkle proof, or receipt, over the entry produced in a Confidential Ledger to verify that the write operation was correctly saved. A write transaction receipt is proof that the system has committed the corresponding transaction and can be used to verify that the entry has been effectively appended to the ledger.Please refer to the followingarticlefor more information about Azure Confidential Ledger write transaction receipts.Receipt VerificationAfter getting a receipt for a write transaction, Azure Confidential Ledger users can verify the contents of the fetched receipt following a verification algorithm. The success of the verification is proof that the write operation associated to the receipt was correctly appended into the immutable ledger.Please refer to the followingarticlefor more information about the verification process for Azure Confidential Ledger write transaction receipts.Application ClaimsAzure Confidential Ledger applications can attach arbitrary data, called application claims, to write transactions. These claims represent the actions executed during a write operation. When attached to a transaction, the SHA-256 digest of the claims object is included in the ledger and committed as part of the write transaction. This guarantees that the digest is signed in place and cannot be tampered with.Later, application claims can be revealed in their un-digested form in the receipt payload corresponding to the same transaction where they were added. This allows users to leverage the information in the receipt to re-compute the same claims digest that was attached and signed in place by the Azure Confidential Ledger instance during the transaction. The claims digest can be used as part of the write transaction receipt verification process, providing an offline way for users to fully verify the authenticity of the recorded claims.More details on the application claims format and the digest computation algorithm can be found at the following links:Azure Confidential Ledger application claimsAzure Confidential Ledger application claims digest verificationPlease refer to the following CCF documentation pages for more information about CCF Application claims:Application ClaimsUser-Defined Claims in ReceiptsConfidential computingAzure Confidential Computingallows you to isolate and protect your data while it is being processed in the cloud. Azure Confidential Ledger runs on Azure Confidential Computing virtual machines, thus providing stronger data protection with encryption of data in use.Confidential Consortium FrameworkAzure Confidential Ledger is built on Microsoft Research's open-sourceConfidential Consortium Framework (CCF). Under CCF, applications are managed by a consortium of members with the ability to submit proposals to modify and govern application operation. In Azure Confidential Ledger, Microsoft Azure owns an operator member identity that allows it to perform governance and maintenance actions like replacing unhealthy nodes in the Confidential Ledger and upgrading the enclave code.ExamplesThis section contains code snippets covering common tasks, including:Append entryRetrieving ledger entriesMaking a ranged queryManaging usersUsing certificate authenticationVerify write transaction receiptsAppend entryData that needs to be stored immutably in a tamper-proof manner can be saved to Azure Confidential Ledger by appending an entry to the ledger.Since Confidential Ledger is a distributed system, rare transient failures may cause writes to be lost. For entries that must be preserved, it is advisable to verify that the write became durable. For less important writes where higher client throughput is preferred, the wait step may be skipped.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)post_entry_result=ledger_client.create_ledger_entry({"contents":"Hello world!"})transaction_id=post_entry_result["transactionId"]wait_poller=ledger_client.begin_wait_for_commit(transaction_id)wait_poller.wait()print(f'Ledger entry at transaction id{transaction_id}has been committed successfully')Alternatively, the client may wait for commit when writing a ledger entry.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)post_poller=ledger_client.begin_create_ledger_entry({"contents":"Hello world again!"})new_post_result=post_poller.result()print('The new ledger entry has been committed successfully at transaction id 'f'{new_post_result["transactionId"]}')Retrieving ledger entriesGetting ledger entries older than the latest may take some time as the service is loading historical entries, so a poller is provided.Ledger entries are retrieved by collection. The returned value is the value contained in the specified collection at the point in time identified by the transaction id.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)post_poller=ledger_client.begin_create_ledger_entry({"contents":"Original hello"})post_result=post_poller.result()post_transaction_id=post_result["transactionId"]latest_entry=ledger_client.get_current_ledger_entry()print(f'Current entry (transaction id ={latest_entry["transactionId"]}) 'f'in collection{latest_entry["collectionId"]}:{latest_entry["contents"]}')post_poller=ledger_client.begin_create_ledger_entry({"contents":"Hello!"})post_result=post_poller.result()get_entry_poller=ledger_client.begin_get_ledger_entry(post_transaction_id)older_entry=get_entry_poller.result()print(f'Contents of{older_entry["entry"]["collectionId"]}at{post_transaction_id}:{older_entry["entry"]["contents"]}')Making a ranged queryLedger entries may be retrieved over a range of transaction ids. Entries will only be returned from the default or specified collection.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)post_poller=ledger_client.begin_create_ledger_entry({"contents":"First message"})first_transaction_id=post_poller.result()["transactionId"]foriinrange(10):ledger_client.create_ledger_entry({"contents":f"Message{i}"})post_poller=ledger_client.begin_create_ledger_entry({"contents":"Last message"})last_transaction_id=post_poller.result()["transactionId"]ranged_result=ledger_client.list_ledger_entries(from_transaction_id=first_transaction_id,to_transaction_id=last_transaction_id,)forentryinranged_result:print(f'Contents at{entry["transactionId"]}:{entry["contents"]}')Managing usersUsers withAdministratorprivileges can manage users of the Confidential Ledger directly with the Confidential Ledger itself. Available roles areReader(read-only),Contributor(read and write), andAdministrator(read, write, and add or remove users).fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)user_id="some AAD object id"user=ledger_client.create_or_update_user(user_id,{"assignedRole":"Contributor"})# A client may now be created and used with AAD credentials (i.e. AAD-issued JWT tokens) for the user identified by `user_id`.user=ledger_client.get_user(user_id)assertuser["userId"]==user_idassertuser["assignedRole"]=="Contributor"ledger_client.delete_user(user_id)# For a certificate-based user, their user ID is the fingerprint for their PEM certificate.user_id="PEM certificate fingerprint"user=ledger_client.create_or_update_user(user_id,{"assignedRole":"Reader"})user=ledger_client.get_user(user_id)assertuser["userId"]==user_idassertuser["assignedRole"]=="Reader"ledger_client.delete_user(user_id)Using certificate authenticationClients may authenticate with a client certificate in mutual TLS instead of via an Azure Active Directory token.ConfidentialLedgerCertificateCredentialis provided for such clients.fromazure.confidentialledgerimport(ConfidentialLedgerCertificateCredential,ConfidentialLedgerClient,)fromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=ConfidentialLedgerCertificateCredential(certificate_path="Path to user certificate PEM file")ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)Verify write transaction receiptsClients can leverage the receipt verification library in the SDK to verify write transaction receipts issued by Azure Confidential Legder instances. The utility can be used to fully verify receipts offline as the verification algorithm does not require to be connected to a Confidential ledger or any other Azure service.Once a new entry has been appended to the ledger (please refer tothis example), it is possible to get a receipt for the committed write transaction.fromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredential# Replace this with the Confidential Ledger IDledger_id="my-ledger-id"# Setup authenticationcredential=DefaultAzureCredential()# Create a Ledger Certificate client and use it to# retrieve the service identity for our ledgeridentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id=ledger_id)# Save ledger service certificate into a file for later useledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])# Create Confidential Ledger clientledger_client=ConfidentialLedgerClient(endpoint=f"https://{ledger_id}.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)# The method begin_get_receipt returns a poller that# we can use to wait for the receipt to be available for retrievalget_receipt_poller=ledger_client.begin_get_receipt(transaction_id)get_receipt_result=get_receipt_poller.result()print(f"Write receipt for transaction id{transaction_id}was successfully retrieved:{get_receipt_result}")After fetching a receipt for a write transaction, it is possible to call theverify_receiptfunction to verify that the receipt is valid. The function can accept an optional list of application claims to verify against the receipt claims digest.fromazure.confidentialledger.receiptimport(verify_receipt,)# Read contents of service certificate file saved in previous step.withopen(ledger_tls_cert_file_name,"r")asservice_cert_file:service_cert_content=service_cert_file.read()# Optionally read application claims, if anyapplication_claims=get_receipt_result.get("applicationClaims",None)try:# Verify the contents of the receipt.verify_receipt(get_receipt_result["receipt"],service_cert_content,application_claims=application_claims)print(f"Receipt for transaction id{transaction_id}successfully verified")exceptValueError:print(f"Receipt verification for transaction id{transaction_id}failed")A full sample Python program that shows how to append a new entry to a running Confidential Ledger instance, get a receipt for the committed transaction, and verify the receipt contents can be found under thesamplesfolder:get_and_verify_receipt.py.Async APIThis library includes a complete async API supported on Python 3.5+. To use it, you must first install an async transport, such asaiohttp. See theazure-core documentationfor more information.An async client is obtained fromazure.confidentialledger.aio. Methods have the same names and signatures as the synchronous client. Samples may be foundhere.TroubleshootingGeneralConfidential Ledger clients raise exceptions defined inazure-core. For example, if you try to get a transaction that doesn't exist,ConfidentialLedgerClientraisesResourceNotFoundError:fromazure.core.exceptionsimportResourceNotFoundErrorfromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredentialidentity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name)try:ledger_client.begin_get_ledger_entry(transaction_id="10000.100000"# Using a very high id that probably doesn't exist in the ledger if it's relatively new.)exceptResourceNotFoundErrorase:print(e.message)LoggingThis library uses the standardlogginglibrary for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with thelogging_enableargument:importloggingimportsysfromazure.confidentialledgerimportConfidentialLedgerClientfromazure.confidentialledger.certificateimportConfidentialLedgerCertificateClientfromazure.identityimportDefaultAzureCredential# Create a logger for the 'azure' SDKlogger=logging.getLogger('azure')logger.setLevel(logging.DEBUG)# Configure a console outputhandler=logging.StreamHandler(stream=sys.stdout)logger.addHandler(handler)identity_client=ConfidentialLedgerCertificateClient()network_identity=identity_client.get_ledger_identity(ledger_id="my-ledger-id")ledger_tls_cert_file_name="ledger_certificate.pem"withopen(ledger_tls_cert_file_name,"w")ascert_file:cert_file.write(network_identity["ledgerTlsCertificate"])credential=DefaultAzureCredential()# This client will log detailed information about its HTTP sessions, at DEBUG level.ledger_client=ConfidentialLedgerClient(endpoint="https://my-ledger-id.confidential-ledger.azure.com",credential=credential,ledger_certificate_path=ledger_tls_cert_file_name,logging_enable=True,)Similarly,logging_enablecan enable detailed logging for a single operation, even when it isn't enabled for the client:ledger_client.get_current_ledger_entry(logging_enable=True)Next stepsMore sample codeThese code samples show common scenario operations with the Azure Confidential Ledger client library.Common scenariosWriting to the ledger:write_to_ledger.pywrite_to_ledger_async.py(async version)Write many ledger entries and retrieve them all afterwards:list_ledger_entries.pylist_ledger_entries_async.py(async version)Manage users using service-implemented role-based access control:manage_users.pymanage_users_async.py(async version)Advanced scenariosUsing collections:use_collections.pyuse_collections_async.py(async version)Getting receipts for ledger writes:get_receipt.pyget_receipt_async.py(async version)Verifying service details:verify_service.pyverify_service_async.py(async version)Additional DocumentationFor more extensive documentation on Azure Confidential Ledger, see theAPI reference documentation. You may also read more about Microsoft Research's open-sourceConfidential Consortium Framework.ContributingThis project welcomes contributions and suggestions. Most contributions require
you to agree to a Contributor License Agreement (CLA) declaring that you have
the right to, and actually do, grant us the rights to use your contribution.
For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether
you need to provide a CLA and decorate the PR appropriately (e.g., label,
comment). Simply follow the instructions provided by the bot. You will only
need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information, see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.Release History1.1.1 (2023-08-01)Bugs FixedAllow someResourceNotFoundErroroccurrences inbegin_wait_for_committo account for unexpected loss of session stickiness. These errors may occur when the connected node changes and transactions have not been fully replicated.1.1.0 (2023-05-09)Features AddedAddazure.confidentialledger.receiptmodule for Azure Confidential Ledger write transaction receipt verification.Addverify_receiptfunction to verify write transaction receipts from a receipt JSON object. The function accepts an optional, keyword-only, list of application claims parameter, which can be used to compute the claims digest from the given claims: the verification would fail if the computed digest value does not match theclaimsDigestvalue present in the receipt.Addcompute_claims_digestfunction to compute the claims digest from a list of application claims JSON objects.Add sample code to get and verify a write receipt from a running Confidential Ledger instance.Update README with examples and documentation for receipt verification and application claims.Other ChangesAdd dependency on Pythoncryptographylibrary (>= 2.1.4)Add tests for receipt verification models and receipt verification public method.Add tests for application claims models and digest computation public method.1.0.0 (2022-07-19)GA Data Plane Python SDK for Confidential Ledger.Bugs FixedUser ids that are certificate fingerprints are no longer URL-encoded in the request URI.Breaking ChangesRemoved all models. Methods now return JSON directly.sub_ledger_idfields are now namedcollection_id.azure.confidentialledger.identity_servicehas been renamed toazure.confidentialledger.certificate.ConfidentialLedgerIdentityServiceClientis nowConfidentialLedgerCertificateClient.post_ledger_entryhas been renamed tocreate_ledger_entry.Other ChangesPython 2.7 is no longer supported. Please use Python version 3.7 or later.Convenience poller methods added for certain long-running operations.Add new supported API version:2022-05-13.1.0.0b1 (2021-05-12)This is the initial release of the Azure Confidential Ledger library.