Package: azure-storage-queue
Azure Storage Queues client library for PythonAzure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. A single queue message can be up to 64 KiB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.Common uses of Queue storage include:Creating a backlog of work to process asynchronouslyPassing messages between different parts of a distributed applicationSource code|Package (PyPI)|Package (Conda)|API reference documentation|Product documentation|SamplesGetting startedPrerequisitesPython 3.7 or later is required to use this package. For more details, please read our page onAzure SDK for Python version support policy.You must have anAzure subscriptionand anAzure storage accountto use this package.Install the packageInstall the Azure Storage Queues client library for Python withpip:pipinstallazure-storage-queueCreate a storage accountIf you wish to create a new storage account, you can use theAzure Portal,Azure PowerShell,
orAzure CLI:# Create a new resource group to hold the storage account -# if using an existing resource group, skip this stepazgroupcreate--namemy-resource-group--locationwestus2# Create the storage accountazstorageaccountcreate-nmy-storage-account-name-gmy-resource-groupCreate the clientThe Azure Storage Queues client library for Python allows you to interact with three types of resources: the storage
account itself, queues, and messages. Interaction with these resources starts with an instance of aclient.
To create a client object, you will need the storage account's queue service endpoint URL and a credential that allows
you to access the storage account:fromazure.storage.queueimportQueueServiceClientservice=QueueServiceClient(account_url="https://<my-storage-account-name>.queue.core.windows.net/",credential=credential)Looking up the account URLYou can find the storage account's queue service URL using theAzure Portal,Azure PowerShell,
orAzure CLI:# Get the queue service URL for the storage accountazstorageaccountshow-nmy-storage-account-name-gmy-resource-group--query"primaryEndpoints.queue"Types of credentialsThecredentialparameter may be provided in a number of different forms, depending on the type ofauthorizationyou wish to use:To use ashared access signature (SAS) token,
provide the token as a string. If your account URL includes the SAS token, omit the credential parameter.
You can generate a SAS token from the Azure Portal under "Shared access signature" or use one of thegenerate_sas()functions to create a sas token for the storage account or queue:fromdatetimeimportdatetime,timedeltafromazure.storage.queueimportQueueServiceClient,generate_account_sas,ResourceTypes,AccountSasPermissionssas_token=generate_account_sas(account_name="<storage-account-name>",account_key="<account-access-key>",resource_types=ResourceTypes(service=True),permission=AccountSasPermissions(read=True),start=datetime.utcnow(),expiry=datetime.utcnow()+timedelta(hours=1))queue_service_client=QueueServiceClient(account_url="https://<my_account_name>.queue.core.windows.net",credential=sas_token)To use a storage accountshared key(aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys"
section or by running the following Azure CLI command:az storage account keys list -g MyResourceGroup -n MyStorageAccountUse the key as the credential parameter to authenticate the client:fromazure.storage.queueimportQueueServiceClientservice=QueueServiceClient(account_url="https://<my_account_name>.queue.core.windows.net",credential="<account_access_key>")To use anAzure Active Directory (AAD) token credential,
provide an instance of the desired credential type obtained from theazure-identitylibrary.
For example,DefaultAzureCredentialcan be used to authenticate the client.This requires some initial setup:Install azure-identityRegister a new AAD applicationand give permissions to access Azure StorageGrant accessto Azure Queue data with RBAC in the Azure PortalSet the values of the client ID, tenant ID, and client secret of the AAD application as environment variables:
AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRETUse the returned token credential to authenticate the client:fromazure.identityimportDefaultAzureCredentialfromazure.storage.queueimportQueueServiceClienttoken_credential=DefaultAzureCredential()queue_service_client=QueueServiceClient(account_url="https://<my_account_name>.queue.core.windows.net",credential=token_credential)Creating the client from a connection stringDepending on your use case and authorization method, you may prefer to initialize a client instance with a storage
connection string instead of providing the account URL and credential separately. To do this, pass the storage
connection string to the client'sfrom_connection_stringclass method:fromazure.storage.queueimportQueueServiceClientconnection_string="DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net"service=QueueServiceClient.from_connection_string(conn_str=connection_string)The connection string to your storage account can be found in the Azure Portal under the "Access Keys" section or by running the following CLI command:azstorageaccountshow-connection-string-gMyResourceGroup-nMyStorageAccountKey conceptsThe following components make up the Azure Queue Service:The storage account itselfA queue within the storage account, which contains a set of messagesA message within a queue, in any format, of up to 64 KiBThe Azure Storage Queues client library for Python allows you to interact with each of these components through the
use of a dedicated client object.Async ClientsThis library includes a complete async API supported on Python 3.5+. To use it, you must
first install an async transport, such asaiohttp.
Seeazure-core documentationfor more information.Async clients and credentials should be closed when they're no longer needed. These
objects are async context managers and define asyncclosemethods.ClientsTwo different clients are provided to interact with the various components of the Queue Service:QueueServiceClient-
this client represents interaction with the Azure storage account itself, and allows you to acquire preconfigured
client instances to access the queues within. It provides operations to retrieve and configure the account
properties as well as list, create, and delete queues within the account. To perform operations on a specific queue,
retrieve a client using theget_queue_clientmethod.QueueClient-
this client represents interaction with a specific queue (which need not exist yet). It provides operations to
create, delete, or configure a queue and includes operations to send, receive, peek, delete, and update messages
within it.MessagesSend- Adds a message to the queue and optionally sets a visibility timeout for the message.Receive- Retrieves a message from the queue and makes it invisible to other consumers.Peek- Retrieves a message from the front of the queue, without changing the message visibility.Update- Updates the visibility timeout of a message and/or the message contents.Delete- Deletes a specified message from the queue.Clear- Clears all messages from the queue.ExamplesThe following sections provide several code snippets covering some of the most common Storage Queue tasks, including:Creating a queueSending messagesReceiving messagesCreating a queueCreate a queue in your storage accountfromazure.storage.queueimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")queue.create_queue()Use the async client to create a queuefromazure.storage.queue.aioimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")awaitqueue.create_queue()Sending messagesSend messages to your queuefromazure.storage.queueimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")queue.send_message("I'm using queues!")queue.send_message("This is my second message")Send messages asynchronouslyimportasynciofromazure.storage.queue.aioimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")awaitasyncio.gather(queue.send_message("I'm using queues!"),queue.send_message("This is my second message"))Receiving messagesReceive and process messages from your queuefromazure.storage.queueimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")response=queue.receive_messages()formessageinresponse:print(message.content)queue.delete_message(message)# Printed messages from the front of the queue:# >> I'm using queues!# >> This is my second messageReceive and process messages in batchesfromazure.storage.queueimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")response=queue.receive_messages(messages_per_page=10)formessage_batchinresponse.by_page():formessageinmessage_batch:print(message.content)queue.delete_message(message)Receive and process messages asynchronouslyfromazure.storage.queue.aioimportQueueClientqueue=QueueClient.from_connection_string(conn_str="<connection_string>",queue_name="myqueue")response=queue.receive_messages()asyncformessageinresponse:print(message.content)awaitqueue.delete_message(message)Optional ConfigurationOptional keyword arguments that can be passed in at the client and per-operation level.Retry Policy configurationUse the following keyword arguments when instantiating a client to configure the retry policy:retry_total(int): Total number of retries to allow. Takes precedence over other counts.
Pass inretry_total=0if you do not want to retry on requests. Defaults to 10.retry_connect(int): How many connection-related errors to retry on. Defaults to 3.retry_read(int): How many times to retry on read errors. Defaults to 3.retry_status(int): How many times to retry on bad status codes. Defaults to 3.retry_to_secondary(bool): Whether the request should be retried to secondary, if able.
This should only be enabled of RA-GRS accounts are used and potentially stale data can be handled.
Defaults toFalse.Other client / per-operation configurationOther optional configuration keyword arguments that can be specified on the client or per-operation.Client keyword arguments:connection_timeout(int): The number of seconds the client will wait to establish a connection to the server.
Defaults to 20 seconds.read_timeout(int): The number of seconds the client will wait, between consecutive read operations, for a
response from the server. This is a socket level timeout and is not affected by overall data size. Client-side read
timeouts will be automatically retried. Defaults to 60 seconds.transport(Any): User-provided transport to send the HTTP request.Per-operation keyword arguments:raw_response_hook(callable): The given callback uses the response returned from the service.raw_request_hook(callable): The given callback uses the request before being sent to service.client_request_id(str): Optional user specified identification of the request.user_agent(str): Appends the custom value to the user-agent header to be sent with the request.logging_enable(bool): Enables logging at the DEBUG level. Defaults to False. Can also be passed in at
the client level to enable it for all requests.logging_body(bool): Enables logging the request and response body. Defaults to False. Can also be passed in at
the client level to enable it for all requests.headers(dict): Pass in custom headers as key, value pairs. E.g.headers={'CustomValue': value}TroubleshootingGeneralStorage Queue clients raise exceptions defined inAzure Core.This list can be used for reference to catch thrown exceptions. To get the specific error code of the exception, use theerror_codeattribute, i.e,exception.error_code.LoggingThis library uses the standardlogginglibrary for logging.
Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
level.Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with thelogging_enableargument:importsysimportloggingfromazure.storage.queueimportQueueServiceClient# Create a logger for the 'azure.storage.queue' SDKlogger=logging.getLogger('azure.storage.queue')logger.setLevel(logging.DEBUG)# Configure a console outputhandler=logging.StreamHandler(stream=sys.stdout)logger.addHandler(handler)# This client will log detailed information about its HTTP sessions, at DEBUG levelservice_client=QueueServiceClient.from_connection_string("your_connection_string",logging_enable=True)Similarly,logging_enablecan enable detailed logging for a single operation,
even when it isn't enabled for the client:service_client.get_service_stats(logging_enable=True)Next stepsMore sample codeGet started with ourQueue samples.Several Storage Queues Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Storage Queues:queue_samples_hello_world.py(async version) - Examples found in this article:Client creationCreate a queueSend messagesReceive messagesqueue_samples_authentication.py(async version) - Examples for authenticating and creating the client:From a connection stringFrom a shared access keyFrom a shared access signature tokenFrom Azure Active Directoryqueue_samples_service.py(async version) - Examples for interacting with the queue service:Get and set service propertiesList queues in a storage accountCreate and delete a queue from the serviceGet the QueueClientqueue_samples_message.py(async version) - Examples for working with queues and messages:Set an access policyGet and set queue metadataSend and receive messagesDelete specified messages and clear all messagesPeek and update messagesAdditional documentationFor more extensive documentation on Azure Queue storage, see theAzure Queue storage documentationon docs.microsoft.com.ContributingThis project welcomes contributions and suggestions.  Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct. For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.Release History12.9.0 (2023-12-05)Other ChangesUpdated type hints across the entire package and enabled MyPy to run during CI. Some public types may have been adjusted if they were previously erroneous or incomplete.12.8.0 (2023-11-07)Features AddedStable release of features from 12.18.0b112.8.0b1 (2023-10-17)Features AddedAddedaudienceas an optional keyword that can be specified on APIs that have acredentialparameter. This
keyword only has an effect when the credential provided is of typeTokenCredential.12.7.3 (2023-10-10)Bugs FixedFixed an issue when an invalid type was provided forcredentialduring client construction, the__str__of the object would be present in the exception message and therefore potentially logged.12.7.2 (2023-09-25)Bugs FixedFixed an issue whereuser_agentwas being ignored onsend_message,receive_message,receive_messages,update_message, andpeek_messagesif client was configured for encryption.12.7.1 (2023-09-13)Bugs FixedFixed breakingKeyError: 'sdk_moniker'increate_configuration.
NOTE: This is not an exported method and therefore should not be imported/called directly.12.7.0 (2023-09-12)Features AddedStable release of features from 12.7.0b112.7.0b1 (2023-08-08)Other ChangesBumpedazure-coreminimum dependency to 1.28.0 andtyping-extensionsto 4.3.0.12.6.0 (2023-02-22)Features AddedStable release of features from 12.6.0b112.6.0b1 (2023-02-02)Other ChangesRemovedmsrestdependency.Addedtyping-extensions>=4.0.1as a dependency.Addedisodate>=0.6.1as a dependency.Added extra dependencyaiofor installing optional async dependencies. Usepip install azure-storage-queue[aio]to install.12.5.0 (2022-10-11)Features AddedStable release of features from 12.5.0b1.Other ChangesChanged the default value forread_timeoutto 60 seconds for all clients.12.5.0b1 (2022-08-23)This version and all future versions will require Python 3.7+. Python 3.6 is no longer supported.Features AddedAdded support forAzureNamedKeyCredentialas a validcredentialtype.12.4.0 (2022-07-07)Features AddedStable release of features from 12.4.0b1.12.4.0b1 (2022-06-15)Features AddedIntroduced version 2.0 of client-side encryption for Queue messages which utilizes AES-GCM-256 encryption.
Version 1.0 is now deprecated and no longer considered secure. If you are using client-side encryption, it ishighly recommendedthat you update to version 2.0.
The encryption version can be specified on any client constructor via theencryption_versionkeyword (i.e.encryption_version='2.0').12.3.0 (2022-05-09)Features AddedStable release of features from 12.3.0b1.Bugs FixedFixed a bug, introduced in the previous beta release, that caused Authentication errors when attempting to use
an Account SAS with certain service level operations.12.3.0b1 (2022-04-14)Features AddedAdded support formax_messagesinreceive_messages()to specify the maximum number of messages to receive from the queue.Other ChangesUpdated SAS token generation to use the latest supported service version by default. Moving to the latest version
also included a change to how account SAS is generated to reflect a change made to the service in SAS generation for
service version 2020-12-06.Updated documentation forreceive_messages()to explain iterator behavior and life-cycle.Added a sample toqueue_samples_message.py(and async-equivalent) showcasing the use ofmax_messagesinreceive_messages().12.2.0 (2022-03-08)This version and all future versions will require Python 3.6+. Python 2.7 is no longer supported.Bugs FixedUpdateazure-coredependency to avoid inconsistent dependencies from being installed.12.1.6 (2021-04-20)FixesMakeAccountName,AccountKeyetc. in conn_str case insensitiveFixed unclosedThreadPoolExecutor(#8955)12.1.5 (2021-01-13)New featuresAdded support forAzureSasCredentialto allow SAS rotation in long living clients.12.1.4 (2020-11-10)New featureAddedreceive_messageon QueueClient to support receiving one message from queue (#14844, #14762)NotesUpdated dependencyazure-corefrom  azure-core<2.0.0,>=1.6.0 to azure-core<2.0.0,>=1.9.0 to get continuation_token attr on AzureError.12.1.3 (2020-09-10)FixesFixed QueueClient type declaration (#11392).12.1.2NotesUpdated dependency from azure-core<2.0.0,>=1.2.2 to azure-core<2.0.0,>=1.6.012.1.1 (2020-03-10)FixesResponses are always decoded as UTF8NotesTheStorageUserAgentPolicyis now replaced with theUserAgentPolicyfrom azure-core. With this, the custom user agents are now added as a prefix instead of being appended.12.1.0 (2019-12-04)New featuresAll the clients now have aclose()method to close the sockets opened by the client when using without a context manager.12.0.0 (2019-10-31)Breaking changesQueueClientnow accepts onlyaccount_urlwith mandatory a string paramqueue_name.
To use a queue_url, the methodfrom_queue_urlmust be used.set_queue_access_policyhas required parametersigned_identifiers.NoRetrypolicy has been removed. Use keyword argumentretry_total=0for no retries.NoEncodePolicyandNoDecodePolicyhave been removed. Usemessage_encode_policy=Noneandmessage_decode_policy=None.Removed types that were accidentally exposed from two modules. OnlyQueueServiceClientandQueueClientshould be imported from azure.storage.queue.aioSome parameters have become keyword only, rather than positional. Some examples include:loopmax_concurrencyvalidate_contenttimeoutetc.QueueMessagehas had its parameters renamed frominsertion_time,time_next_visible,expiration_timetoinserted_on,next_visible_on,expires_on, respectively.Logginghas been renamed toQueueAnalyticsLogging.enqueue_messageis now calledsend_message.Client and model files have been made internal. Users should import from the top level modulesazure.storage.queueandazure.storage.queue.aioonly.Thegenerate_shared_access_signaturemethods on bothQueueServiceClientandQueueClienthave been replaced by module level functionsgenerate_account_sasandgenerate_queue_sas.get_service_statsnow returns a dictget_service_propertiesnow returns a dict with keys consistent toset_service_propertiesNew featuresResourceTypes, andServicesnow have methodfrom_stringwhich takes parameters as a string.Fixes and improvementsFixed an issue where XML is being double encoded and double decoded.12.0.0b4 (2019-10-08)Breaking changesPermission models.AccountPermissions,QueuePermissionshave been renamed toAccountSasPermissions,QueueSasPermissionsrespectively.enum-like list parameters have been removed from both of them.__add__and__or__methods are removed.max_connectionsis now renamed tomax_concurrency.New featuresAccountSasPermissions,QueueSasPermissionsnow have methodfrom_stringwhich takes parameters as a string.12.0.0b3 (2019-09-10)Dependency updatesAdoptedazure-core1.0.0b3If you later want to revert to previous versions of azure-storage-queue, or another Azure SDK
library requiring azure-core 1.0.0b1 or azure-core 1.0.0b2, you must explicitly install
the specific version of azure-core as well. For example:pip install azure-core==1.0.0b2 azure-storage-queue==12.0.0b212.0.0b2 (2019-08-06)Breaking changesThe behavior of listing operations has been modified:The previousmarkerparameter has been removed.The iterable response object now supports aby_pagefunction that will return a secondary iterator of batches of results. This function supports acontinuation_tokenparameter to replace the previousmarkerparameter.The new listing behaviour is also adopted by thereceive_messagesoperation:The receive operation returns a message iterator as before.The returned iterator supports aby_pageoperation to receive messages in batches.New featuresAdded async APIs to subnamespaceazure.storage.queue.aio.Distributed tracing framework OpenCensus is now supported.Dependency updatesAdoptedazure-core1.0.0b2If you later want to revert to azure-storage-queue 12.0.0b1, or another Azure SDK
library requiring azure-core 1.0.0b1, you must explicitly install azure-core
1.0.0b1 as well. For example:pip install azure-core==1.0.0b1 azure-storage-queue==12.0.0b1Fixes and improvementsGeneral refactor of duplicate and shared code.12.0.0b1 (2019-07-02)Version 12.0.0b1 is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure Storage Queues. For more information about this, and preview releases of other Azure SDK libraries, please visithttps://aka.ms/azure-sdk-preview1-python.Breaking changes: New API designOperations are now scoped to a particular client:QueueServiceClient: This client handles account-level operations. This includes managing service properties and listing the queues within an account.QueueClient: The client handles operations within a particular queue. This includes creating or deleting that queue, as well as enqueueing and dequeueing messages.These clients can be accessed by navigating down the client hierarchy, or instantiated directly using URLs to the resource (account or queue).
For full details on the new API, please see thereference documentation.New message iterator, for receiving messages from a queue in a continuous stream.New underlying REST pipeline implementation, based on the newazure-corelibrary.Client and pipeline configuration is now available via keyword arguments at both the client level, and per-operation. See reference documentation for a full list of optional configuration arguments.Authentication usingazure-identitycredentialssee theAzure Identity documentationfor more informationNew error hierarchy:All service errors will now use the base type:azure.core.exceptions.HttpResponseErrorThe are a couple of specific exception types derived from this base type for common error scenarios:ResourceNotFoundError: The resource (e.g. queue, message) could not be found. Commonly a 404 status code.ResourceExistsError: A resource conflict - commonly caused when attempting to create a resource that already exists.ResourceModifiedError: The resource has been modified (e.g. overwritten) and therefore the current operation is in conflict. Alternatively this may be raised if a condition on the operation is not met.ClientAuthenticationError: Authentication failed.No longer have specific operations forget_metadata- useget_propertiesinstead.No longer have specific operations forexists- useget_propertiesinstead.Operationsget_queue_aclandset_queue_aclhave been renamed toget_queue_access_policyandset_queue_access_policy.Operationput_messagehas been renamed toenqueue_message.Operationget_messageshas been renamed toreceive_messages.2.0.1Updated dependency on azure-storage-common.2.0.0Support for 2018-11-09 REST version.1.4.0azure-storage-nspkg is not installed anymore on Python 3 (PEP420-based namespace package)1.3.0Support for 2018-03-28 REST version. Please see our REST API documentation and blog for information about the related added features.1.2.0rc1Support for 2017-11-09 REST version. Please see our REST API documentation and blog for information about the related added features.Added support for OAuth authentication for HTTPS requests(Please note that this feature is available in preview).1.1.0Support for 2017-07-29 REST version. Please see our REST API documentation and blogs for information about the related added features.Queue messages can now have an arbitrarily large or infinite time to live.Error message now contains the ErrorCode from the x-ms-error-code header value.1.0.0The package has switched from Apache 2.0 to the MIT license.