Package: azure-data-tables
Azure Tables client library for PythonAzure Tables is a NoSQL data storage service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.
Tables scales as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
The Azure Tables client can be used to access Azure Storage or Cosmos accounts. This document coversazure-data-tables.Please note, this package is a replacement forazure-cosmosdb-tableswhich is now deprecated. See themigration guidefor more details.Source code|Package (PyPI)|Package (Conda)|API reference documentation|SamplesGetting startedThe Azure Tables SDK can access an Azure Storage or CosmosDB account.PrerequisitesPython 3.8 or later is required to use this package.You must have anAzure subscriptionand eitheranAzure Storage accountoranAzure Cosmos Account.Create accountTo create a new storage account, you can useAzure Portal,Azure PowerShell, orAzure CLI:To create a new cosmos storage account, you can use theAzure CLIorAzure Portal.Install the packageInstall the Azure Tables client library for Python withpip:pipinstallazure-data-tablesCreate the clientThe Azure Tables library allows you to interact with two types of resources:the tables in your accountthe entities within those tables.
Interaction with these resources starts with an instance of aclient. To create a client object, you will need the account's table service endpoint URL and a credential that allows you to access the account. Theendpointcan be found on the page for your storage account in theAzure Portalunder the "Access Keys" section or by running the following Azure CLI command:# Get the table service URL for the accountazstorageaccountshow-nmystorageaccount-gMyResourceGroup--query"primaryEndpoints.table"Once you have the account URL, it can be used to create the service client:fromazure.data.tablesimportTableServiceClientservice=TableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net/",credential=credential)For more information about table service URL's and how to configure custom domain names for Azure Storage check out theofficial documentationTypes of credentialsThecredentialparameter may be provided in a number of different forms, depending on the type of authorization you wish to use. The Tables library supports the following authorizations:Shared KeyConnection StringShared Access Signature TokenTokenCredential(AAD)(Supported on Storage)Creating the client from a shared keyTo use an accountshared key(aka account key or access key), provide the key as a string. This can be found in your storage account in theAzure Portalunder the "Access Keys" section or by running the following Azure CLI command:azstorageaccountkeyslist-gMyResourceGroup-nMyStorageAccountUse the key as the credential parameter to authenticate the client:fromazure.data.tablesimportTableServiceClientfromazure.core.credentialsimportAzureNamedKeyCredentialcredential=AzureNamedKeyCredential("my_account_name","my_access_key")withTableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net",credential=credential)astable_service_client:properties=table_service_client.get_service_properties()print(f"{properties}")Creating the client from a connection stringDepending on your use case and authorization method, you may prefer to initialize a client instance with a connection string instead of providing the account URL and credential separately. To do this, pass the connection string to the client'sfrom_connection_stringclass method. If the connection string does not specify a fully qualified endpoint URL ("TableEndpoint"), or URL suffix ("EndpointSuffix"), the endpoint will be assumed to be an Azure Storage account, and the URL automatically formatted accordingly.For Tables Storage, the connection string can be found in your storage account in theAzure Portalunder the "Access Keys" section or with the following Azure CLI command:azstorageaccountshow-connection-string-gMyResourceGroup-nMyStorageAccountFor Tables Cosmos, the connection string can be found in your cosmos account in theAzure Portalunder the "Connection Strings" section or with the following Azure CLI command:azcosmosdblist-connection-strings-gMyResourceGroup-nMyCosmosAccountCreate a client from a connection string:fromazure.data.tablesimportTableServiceClientconnection_string="AccountName=<my_account_name>;AccountKey=<my_account_key>;EndpointSuffix=<endpoint_suffix>"withTableServiceClient.from_connection_string(conn_str=connection_string)astable_service_client:properties=table_service_client.get_service_properties()print(f"{properties}")Creating the client from a SAS tokenTo use ashared access signature (SAS) token, provide the token as a string. If your account URL includes the SAS token, omit the credential parameter. You can generate a SAS token from the Azure Portal underShared access signatureor use one of thegenerate_*_sas()functions to create a sas token for the account or table:fromdatetimeimportdatetime,timedeltafromazure.data.tablesimportTableServiceClient,generate_account_sas,ResourceTypes,AccountSasPermissionsfromazure.core.credentialsimportAzureNamedKeyCredential,AzureSasCredentialcredential=AzureNamedKeyCredential("my_account_name","my_access_key")# Create a SAS token to use for authentication of a clientsas_token=generate_account_sas(credential,resource_types=ResourceTypes(service=True),permission=AccountSasPermissions(read=True),expiry=datetime.utcnow()+timedelta(hours=1),)withTableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net",credential=AzureSasCredential(sas_token))astable_service_client:properties=table_service_client.get_service_properties()print(f"{properties}")Creating the client from a TokenCredentialAzure Tables provides integration with Azure Active Directory(Azure AD) for identity-based authentication of requests to the Table service when targeting a Storage endpoint. With Azure AD, you can use role-based access control(RBAC) to grant access to your Azure Table resources to users, groups, or applications.To access a table resource with a TokenCredential, the authenticated identity should have either the "Storage Table Data Contributor" or "Storage Table Data Reader" role.With theazure-identitypackage, you can seamlessly authorize requests in both development and production environments. To learn more about Azure AD integration in Azure Storage, see theazure-identity READMEfromazure.data.tablesimportTableServiceClientfromazure.identityimportDefaultAzureCredentialwithTableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net",credential=DefaultAzureCredential())astable_service_client:properties=table_service_client.get_service_properties()print(f"{properties}")Key conceptsCommon uses of the Table service included:Storing TBs of structured data capable of serving web scale applicationsStoring datasets that do not require complex joins, foreign keys, or stored procedures and can be de-normalized for fast accessQuickly querying data using a clustered indexAccessing data using the OData protocol and LINQ filter expressionsThe following components make up the Azure Tables Service:The accountA table within the account, which contains a set of entitiesAn entity within a table, as a dictionaryThe Azure Tables client library for Python allows you to interact with each of these components through the
use of a dedicated client object.ClientsTwo different clients are provided to interact with the various components of the Table Service:TableServiceClient-Get and set account settingQuery, create, and delete tables within the account.Get aTableClientto access a specific table using theget_table_clientmethod.TableClient-Interacts with a specific table (which need not exist yet).Create, delete, query, and upsert entities within the specified table.Create or delete the specified table itself.EntitiesEntities are similar to rows. An entity has aPartitionKey, aRowKey, and a set of properties. A property is a name value pair, similar to a column. Every entity in a table does not need to have the same properties. Entities can be represented as dictionaries like this as an example:entity={'PartitionKey':'color','RowKey':'brand','text':'Marker','color':'Purple','price':'5'}create_entity- Add an entity to the table.delete_entity- Delete an entity from the table.update_entity- Update an entity's information by either merging or replacing the existing entity.UpdateMode.MERGEwill add new properties to an existing entity it will not delete an existing propertiesUpdateMode.REPLACEwill replace the existing entity with the given one, deleting any existing properties not included in the submitted entityquery_entities- Query existing entities in a table usingOData filters.get_entity- Get a specific entity from a table by partition and row key.upsert_entity- Merge or replace an entity in a table, or if the entity does not exist, inserts the entity.UpdateMode.MERGEwill add new properties to an existing entity it will not delete an existing propertiesUpdateMode.REPLACEwill replace the existing entity with the given one, deleting any existing properties not included in the submitted entityExamplesThe following sections provide several code snippets covering some of the most common Table tasks, including:Creating a tableCreating entitiesQuerying entitiesCreating a tableCreate a table in your account and get aTableClientto perform operations on the newly created table:fromazure.data.tablesimportTableServiceClienttable_service_client=TableServiceClient.from_connection_string(conn_str="<connection_string>")table_name="myTable"table_client=table_service_client.create_table(table_name=table_name)Creating entitiesCreate entities in the table:fromazure.data.tablesimportTableServiceClientfromdatetimeimportdatetimePRODUCT_ID=u'001234'PRODUCT_NAME=u'RedMarker'my_entity={u'PartitionKey':PRODUCT_NAME,u'RowKey':PRODUCT_ID,u'Stock':15,u'Price':9.99,u'Comments':u"great product",u'OnSale':True,u'ReducedPrice':7.99,u'PurchaseDate':datetime(1973,10,4),u'BinaryRepresentation':b'product_name'}table_service_client=TableServiceClient.from_connection_string(conn_str="<connection_string>")table_client=table_service_client.get_table_client(table_name="myTable")entity=table_client.create_entity(entity=my_entity)Querying entitiesQuerying entities in the table:fromazure.data.tablesimportTableClientmy_filter="PartitionKey eq 'RedMarker'"table_client=TableClient.from_connection_string(conn_str="<connection_string>",table_name="myTable")entities=table_client.query_entities(my_filter)forentityinentities:forkeyinentity.keys():print(f"Key:{key}, Value:{entity[key]}")Optional ConfigurationOptional keyword arguments can be passed in at the client and per-operation level. The azure-corereference documentationdescribes available configurations for retries, logging, transport protocols, and more.Retry Policy configurationUse the following keyword arguments when instantiating a client to configure the retry policy:retry_total(int): Total number of retries to allow. Takes precedence over other counts.
Pass inretry_total=0if you do not want to retry on requests. Defaults to 10.retry_connect(int): How many connection-related errors to retry on. Defaults to 3.retry_read(int): How many times to retry on read errors. Defaults to 3.retry_status(int): How many times to retry on bad status codes. Defaults to 3.retry_to_secondary(bool): Whether the request should be retried to secondary, if able.
This should only be enabled of RA-GRS accounts are used and potentially stale data can be handled.
Defaults toFalse.Other client / per-operation configurationOther optional configuration keyword arguments that can be specified on the client or per-operation.Client keyword arguments:connection_timeout(int): Optionally sets the connect and read timeout value, in seconds.transport(Any): User-provided transport to send the HTTP request.Per-operation keyword arguments:raw_response_hook(callable): The given callback uses the response returned from the service.raw_request_hook(callable): The given callback uses the request before being sent to service.client_request_id(str): Optional user specified identification of the request.user_agent(str): Appends the custom value to the user-agent header to be sent with the request.logging_enable(bool): Enables logging at the DEBUG level. Defaults to False. Can also be passed in at
the client level to enable it for all requests.headers(dict): Pass in custom headers as key, value pairs. E.g.headers={'CustomValue': value}TroubleshootingGeneralAzure Tables clients raise exceptions defined inAzure Core.
When you interact with the Azure table library using the Python SDK, errors returned by the service respond ot the same HTTP status codes forREST APIrequests. The Table service operations will throw aHttpResponseErroron failure with helpfulerror codes.For examples, if you try to create a table that already exists, a409error is returned indicating "Conflict".fromazure.data.tablesimportTableServiceClientfromazure.core.exceptionsimportHttpResponseErrortable_name='YourTableName'service_client=TableServiceClient.from_connection_string(connection_string)# Create the table if it does not already existtc=service_client.create_table_if_not_exists(table_name)try:service_client.create_table(table_name)exceptHttpResponseError:print(f"Table with name{table_name}already exists")LoggingThis library uses the standardlogginglibrary for logging.
Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
level.Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with thelogging_enableargument:importsysimportloggingfromazure.data.tablesimportTableServiceClient# Create a logger for the 'azure' SDKlogger=logging.getLogger('azure')logger.setLevel(logging.DEBUG)# Configure a console outputhandler=logging.StreamHandler(stream=sys.stdout)logger.addHandler(handler)# This client will log detailed information about its HTTP sessions, at DEBUG levelservice_client=TableServiceClient.from_connection_string("your_connection_string",logging_enable=True)Similarly,logging_enablecan enable detailed logging for a single operation,
even when it is not enabled for the client:service_client.create_entity(entity=my_entity,logging_enable=True)Next stepsGet started with ourTable samples.Several Azure Tables Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Tables.Common ScenariosThese code samples show common scenario operations with the Azure Tables client library. The async versions of the samples (the python sample files appended with _async) show asynchronous operations.Create and delete tables:sample_create_delete_table.py(async version)List and query tables:sample_query_tables.py(async version)Insert and delete entities:sample_insert_delete_entities.py(async version)Query and list entities:sample_query_table.py(async version)Update, upsert, and merge entities:sample_update_upsert_merge_entities.py(async version)Committing many requests in a single transaction:sample_batching.py(async version)Additional documentationFor more extensive documentation on Azure Tables, see theAzure Tables documentationon docs.microsoft.com.Known IssuesA list of currently known issues relating to Cosmos DB table endpoints can be foundhere.ContributingThis project welcomes contributions and suggestions.  Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct. For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.Release History12.5.0 (2024-01-10)Bugs FixedFixed issue in serializing EntityProperty tuples where a value of None could be serialized as the string "None".Other ChangesRefactored batching code to use latest Core models and improve typing.Added a public typeEntityMetadata, it is used inTableEntity's metadata.Added support for Python 3.12.Python 3.7 is no longer supported. Please use Python version 3.8 or later.12.4.4 (2023-09-14)Features AddedEnabled to specify resource typecontainerin account SAS access.Bugs FixedFixed a bug when submitting transactions with an empty operation list. (#31471)Fixed a bug when decoding response body in string type. Thanks @kldtz for the contribution! (#31265)Fixed a bug when retrieving an entity with partition key and/or row key in empty string, the empty string values were disappeared in result. (#31920)Other ChangesBumped minimum dependency onazure-coreto>=1.29.4. (#28918#31471)12.4.3 (2023-06-13)Bugs FixedFixed a bug in getting error attribute values when operations failed. (#27410)Other ChangesAdjusted dependency onisodateto<1.0.0,>=0.6.1.12.4.2 (2023-02-07)Bugs FixedFixed a bug when deleting an entity with partition key or row key in empty string.(#24480)Other ChangesAdded support for Python 3.11.Droppedmsrestrequirement.Added dependencyisodatewith version range>=0.6.0(isodatewas required bymsrest).Added dependencytyping-extensionswith version range>=4.3.0.12.4.1 (2022-10-11)Bugs FixedFix handling of client-side exceptions that get raised during service requests (such as#21416) (#24788)Other ChangesPython 3.6 is no longer supported. Please use Python version 3.7 or later.Bumped minimum dependency onazure-coreto>=1.24.0.Bumped minimum dependency onmsrestto>=0.7.1.Added dependencyyarlwith version range<2.0,>=1.0.12.4.0 (2022-05-10)Features AddedSupport for multitenant authentication (#24278)Bugs FixedFixed bug where odmtype tag was not being included for boolean and int32 types even when a full EdmProperty tuple was passed in. This is needed for CLI compatibility.12.3.0 (2022-03-10)Bugs FixedValidation of the table name has been removed from the constructor of the TableClient. Instead individual APIs will validate the table name and raise a ValueError only if the service rejects the request due to the table name not being valid (#23106)Fixed hard-coded URL scheme in batch requests (#21953)Improved documentation for query formatting inquery_entitiesAPIs (#23235)Removed unsecure debug loggingOther ChangesPython 2.7 is no longer supported. Please use Python version 3.6 or later.Bumped dependency onazure-coreto>=1.15.012.2.0 (2021-11-10)WarningThis release involves a bug fix that may change the behaviour for some users. Partition and Row keys that contain a single quote character (') will now be automatically escaped for upsert, update and delete entity operations. Partition and Row keys that were already escaped, or contained duplicate single quote char ('') will now be treated as unescaped values.Bugs FixedResolved bug where strings couldn't be used instead of enum value for entity Update Mode (#20247).Resolved bug where single quote characters in Partition and Row keys were not escaped correctly (#20301).Features AddedAdded support for async iterators in `aio.TableClient.submit_transaction (#21083, thank you yashbhutoria).Other ChangesBumped dependency onmsrestto>=0.6.2112.1.0 (2021-07-06)Features AddedStorage Accounts only:TableClientandTableServiceClients can now useazure-identitycredentials for authentication. Note: ATableClientauthenticated with aTokenCredentialcannot use theget_table_access_policyorset_table_access_policymethods.12.0.0 (2021-06-08)BreakingEdmType.Binary data in entities will now be deserialized asbytesin Python 3 andstrin Python 2, rather than anEdmPropertyinstance. Likewise on serialization,bytesin Python 3 andstrin Python 2 will be interpreted as binary (this is unchanged for Python 3, but breaking for Python 2, wherestrwas previously serialized as EdmType.String)TableClient.create_tablenow returns an instance ofTableItem.All optional parameters for model constructors are now keyword-only.Storage service configuration models have now been prefixed withTable, includingTableAccessPolicy,TableMetrics,TableRetentionPolicy,TableCorsRuleAll parameters forTableServiceClient.set_service_propertiesare now keyword-only.Thecredentialparameter for all Clients is now keyword-only.The methodTableClient.get_access_policywill now returnNonewhere previously it returned an "empty" access policy object.Timestamp properties onTableAccessPolicyinstances returned fromTableClient.get_access_policywill now be deserialized todatetimeinstances.FixesFixed support for Cosmos emulator endpoint, via URL/credential or connection string.Fixed table name from URL parsing inTableClient.from_table_urlclassmethod.Theaccount_nameattribute on clients will now be pulled from anAzureNamedKeyCredentialif used.Any additional odata metadata is returned in entity's metadata.The timestamp in entity metadata is now deserialized to a timestamp.If thepreferheader is added in thecreate_entityoperation, the echo will be returned.Errors raised on a 412 if-not-match error will now be a specificazure.core.exceptions.ResourceModifiedError.EdmType.DOUBLEvalues are now explicitly typed in the request payload.Fixed de/serialization of list attributes onTableCorsRule.12.0.0b7 (2021-05-11)BreakingTheaccount_urlparameter in the client constructors has been renamed toendpoint.TheTableEntityobject now acts exclusively like a dictionary, and no longer supports key access via attributes.Metadata of an entity is now accessed viaTableEntity.metadataattribute rather than a method.Removed explicitLinearRetryandExponentialRetryin favor of keyword parameter.Renamedfilterparameter in query APIs toquery_filter.Thelocation_modeattribute on clients is now read-only. This has been added as a keyword parameter to the constructor.TheTableItem.table_namehas been renamed toTableItem.name.Removed theTableClient.create_batchmethod along with theTableBatchOperationsobject. The transactional batching is now supported via a simple Python list of tuples.TableClient.send_batchhas been renamed toTableClient.submit_transaction.RemovedBatchTransactionResultobject in favor of returning an iterable of batched entities with returned metadata.Removed Batching context-manager behaviorEntityPropertyis now a NampedTuple, and can be represented by a tuple of(entity, EdmType).RenamedEntityProperty.typetoEntityProperty.edm_type.BatchErrorExceptionhas been renamed toTableTransactionError.Thelocation_modeis no longer a public attribute on the Clients.The only supported credentials areAzureNamedKeyCredential,AzureSasCredential, or authentication by connection stringRemoveddateandapi_versionfrom theTableItemclass.FixesFixed issue with Cosmos merge operations.Removed legacy Storage policies from pipeline.Removed unused legacy client-side encryption attributes from client classes.Fixed sharing of pipeline between service/table clients.Added support for Azurite storage emulatorThrows aRequestTooLargeErroron transaction requests that return a 413 error codeAdded support for Int64 and Binary types in query filtersAdded support forselectkeyword parameter toTableClient.get_entity().Onupdate_entityanddelete_entityif noetagis supplied via kwargs, theetagin the entity will be used if it is in the entity.12.0.0b6 (2021-04-06)Updated deserialization of datetime fields in entities to support preservation of the service format with additional decimal place.Passing a string parameter into a query filter will now be escaped to protect against injection.Fixed bug in incrementing retries in async retry policy12.0.0b5 (2021-03-09)This version and all future versions will require Python 2.7 or Python 3.6+, Python 3.5 is no longer supported.Adds SAS credential as an authentication optionBumps minimum requirement ofazure-coreto 1.10.0Bumped minimum requirement of msrest from0.6.10to0.6.19.Adds support for datetime entities with millisecondsAdds support for Shared Access Signature authentication12.0.0b4 (2021-01-12)Fixes anissuewherequery_entitieskwargparameterswould not work with multiple parameters or with non-string parameters. This now works with multiple parameters and numeric, string, boolean, UUID, and datetime objects.Fixes anissuewheredelete_entitywill return aClientAuthenticationErrorwhen the '@' symbol is included in the entity.12.0.0b3 (2020-11-12)Add support for transactional batching of entity operations.Fixed deserialization bug inlist_tablesandquery_tableswhereTableItem.table_namewas an object instead of a string.Fixed issue where unrecognized entity data fields were silently ignored. They will now raise aTypeError.Fixed issue where query filter parameters were being ignored (#15094)12.0.0b2 (2020-10-07)Adds support for Enumerable types by converting the Enum to a string before sending to the service12.0.0b1 (2020-09-08)This is the first beta of theazure-data-tablesclient library. The Azure Tables client library can seamlessly target either Azure Table storage or Azure Cosmos DB table service endpoints with no code changes.