Package: azqueuemanager
AZ Queue ManagerAZ Queue Manager is a tool to manage messages in Azure Storage Queue.It is a command line tool that can be used to send and receive messages from a queue. It can also be used to create and delete queues.Getting StartedInstall AZQueueManager and Extensions# Install the base packagepipinstallazqueuemanager# Install the csv-input extensionpipinstallazqeueuemanager-csv# or some other extensionSetup your scriptIf this is your input filedata.json#data.json[{"id":"msg_1","text":"hello world"},{"id":"msg_2":"text":"hello universe"}]Create your script.# test.pyfromazqueuemanagerimportQueueManager,fromazqueuemanager-jsonimportJSONTransformerWe'll come back to the extension. For now, let's define our queue_client. This is the storage queue itself.# test.pyCONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=MYACCOUNT;AccountKey=MYKEY;EndpointSuffix=core.windows.net"#You'll need your own connection stringqueue_name="MYQUEUENAME"queue=StorageQueue(connection_string,queue_name,)Next we'll need to setup our extension. Every extension has a transformer. That transformer has modules for passing data into and out of the queue. The input transformer will take data and create one or more messages. The output transformer will take a message and create output for each record.These inputs and outputs are pretty dynamic so you will need to look at the documentation for each extension to see what the input and output looks like.The json extension the input transformer takesjson_in_file(orjson_data) and (optionally) ajson_out_file. Thetransform_inwill add messages from the json content. Thetransform_outwill take the messages and create a json file with all the records returned.Let's create an instance of the JSONTransformer and pass it to the QueueManager along with our queue.# test.pyjson_transform=JSONTransformer(json_in_file='data.json',)queue_manager=QueueManager(queue=queueinput_transformer=json_transform,output_transformer=json_transform,)Now we can add or pull data to/from our QueueLoading DataWe pass data in using thequeue_messagesmethod. This will use the data processed from theinput_transformerand add it to the storage queue.If there is noinput_transformerthen thequeue_messagesmethod will take a list of messages (asmessages) and add them to the queue.You can look at all the messages loaded into the queue using thelist_messagesmethod.# test.py# load data into the queuequeue_manager.queue_messages()# list the messages in the queue without popping them from the stackqueue_manager.list_messages()# >>> [#    {id: 1234567890, content:"{'id': 'msg_1', 'text': 'hello world'}"},# ...# ]Previewing DataYou can also preview the next record. Thepreview_messagemethod will return the next message in the queuetransformedwithout removing it from the queue.This is designed to ensure that you're data is correct before pushing it to another service.# test.pyqueue_manager.preview_message()# >>> PREVIEW: message={...} -> transformed_message={'id': 'msg_1', 'text': 'hello world'}Retrieving The DataUsenext_messages(count=n)to retrievenmessages from the queue and process them with theoutput_transformer. By default messages are not deleted from the queue, but you can delete them by settingdelete_after=Truein the method.If you don't add a count, then it will return all the messages in the queue.You can also process the next message ONLY by usingnext_message(delete_after=True). This may be helpful if you want to process the messages one at a time (if the messages have different key:values that may make data more compicated).# test.py# get messages from the queueQueueManager.next_messages()# >>>#	[#		{"id": "msg_1", "text": "hello world"},#		{"id": "msg_2": "text": "hello universe"}#	]