Package: django-sqs
django-sqs==========Author: Maciej Pasternacki <maciej@pasternacki.net>Date: 2010-01-15 FriMaintainer: Adrien Lemaire <lemaire.adrien@gmail.com>Integrate Amazon Simple Queue Service in your Django projectTable of Contents=================1 Setup2 Receivers2.1 Register using a decorator2.2 Register manually2.3 Example receiver3 Receiving4 Sending4.1 Using decorated function4.2 Manually5 Custom message classes5.1 ModelInstanceMessage class6 Management6.1 manage.py sqs_status6.2 manage.py sqs_clear6.3 manage.py sqs_wait7 Views8 FIXME8.1 Sensible forking/threading or multiplexing instead of the fork hack?8.2 Autoimporting receivers.py from apps8.3 docstrings8.4 Minimize polling8.5 Custom exception to leave message in queue1 Setup~~~~~~~~Boto library for accessing Amazon Web Services is required.1. Add `django_sqs' to your Python path2. Add `django_sqs' to INSTALLED_APPS setting3. Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY4. Optionally set SQS_QUEUE_PREFIX to prefix your queues and avoidclashes with other developers, production/staging env and so on.SQS_QUEUE_PREFIX is required when DEBUG is true, and recommendedeven in production mode.5. Optionally set SQS_DEFAULT_VISIBILITY_TIMEOUT (default is 60 seconds)6. Optionally set SQS_POLL_PERIOD (default is 10 seconds)7. Optionally set AWS_REGION (default to "us-east-1")2 Receivers~~~~~~~~~~~~Create receiver function that accepts one argument, which will be aninstance of `boto.sqs.message.Message' or its custom subclass.Then, register it as queue receiver:2.1 Register using a decorator===============================Decorate receiver function with:django_sqs.receiver([queue_name=None, visibility_timeout=None, message_class=None, delete_on_start=False, close_database=False, suffixes=()])Decorated function will become an instance of`django_sqs.registered_queue.RegisteredQueue.ReceiverProxy' class.Instance is callable - you may call it with an instance ofappropriate message class, or its constructor's keyword arguments,and the message will be added to the queue.  Instance provides alsoattributes `direct' with original decorated function, and`registered_queue' with appropriate`django_sqs.registered_queue.RegisteredQueue' instance.If `queue_name' is omitted or false, function's `__module__' and`__name__' is used, with dots converted to double underscores.The `suffixes' argument is a tuple of known queue name suffixes. Ifan unknown suffix is used, a warning will be issued.If `delete_on_start' is true, received message will be deleted fromqueue right after it's been received, before receiver function iscalled.  If it is false (which is the default), it will be deletedafter receiver function has finished, when message has been fullyprocessed.If `close_database' is true, all database connections will beclosed after processing each message to prevent pending unclosedtransactions.Queue name suffixes can be used to split processing similar itemsto multiple queues (e.g. use separate queue for big input items todistribute load).2.2 Register manually======================Alternatively, you can avoid decoration, and register a receivermanually by calling:django_sqs.register(queue_name, [fn=None, visibility_timeout=None, message_class=None, delete_on_start=False, suffixes=()])If `fn' is None or not given, no handler is assigned: messages canbe sent, but won't be received.Create function in modules that will be imported by default(recommendation: use `receivers.py' and import them in `models.py',autoimporting TBD).2.3 Example receiver=====================@receiver("test")def receive_message(msg):print 'received:', msg.get_body()3 Receiving~~~~~~~~~~~~python manage.py runreceiver [--message-limit=N] [--suffix=SUFFIX] [queue_name [queue_name [...]]]If no `queue_name' parameters are given, receive from all configuredqueues.If more than one queue is registered, a new process is forked foreach queue.For each message received on the queue, registered receiver functionis called with the message instance as argument.  If receiverfunction returns successfully, message is then deleted from queue.If receiver message raises an exception, exception traceback islogged using logging module, and message is deleted.  If receiversees a restartable error and wants to keep message in queue, itshould raise `django_sqs.RestartLater' exception - this exception willleave the message in queue.Options:`--message-limit=N': exit after receiving `N' messages`--suffix=SUFFIX': Use queue name suffix4 Sending~~~~~~~~~~4.1 Using decorated function=============================You can simply call function decorated with `@receiver' decorator,providing a message instance or keyword arguments (like for `send'function described below).4.2 Manually=============To send a message manually, use following function:django_sqs.send(queue_name, message=None, suffix=None, **kwargs)`message' should be an instance of `message_class' configured with`receiver' decorator or `register' function for the queue (or`boto.sqs.message.Message').When `message' is omitted or `None', new instance of queue's messageclass will be instantiated using `**kwargs'.  With default messageclass, `boto.sqs.message.Message', we can simply provide body:django_sqs.send("a_queue", body='Lorem ipsum dolor sit amet')`suffix' is a queue name suffix to use.5 Custom message classes~~~~~~~~~~~~~~~~~~~~~~~~~For sending other values than raw, non-unicode strings, any ofclasses provided in `boto.sqs.message' or their subclasses may beused.  The module is well commented (much better than this one), sogo ahead and read the fine source!5.1 ModelInstanceMessage class===============================The `django_sqs.message.ModelInstanceMessage' class is provided forconvenience.  It encodes a single model instance, using Django'sContentType framework (as app/model/primary key triple).  Itaccepts `instance' keyword parameter in constructor, and provides`get_instance()' method.There is no support for passing additional information except theinstance yet.6 Management~~~~~~~~~~~~~6.1 manage.py sqs_status=========================Prints the (approximate) count of messages in the queue.6.2 manage.py sqs_clear========================Clears all queues (by default), or queues named as arguments.Prints number of messages deleted.If queue receivers are running or were running recently, somemessages may be still locked and won't be deleted.  Command mayneed to be re-run.6.3 manage.py sqs_wait=======================Waits until specified (or all) queues are empty.7 Views~~~~~~~~A single view, `django_sqs.views.status', is provided for simple,plain text queue status report (same as `manage.py sqs_status').8 FIXME~~~~~~~~8.1 TODO Sensible forking/threading or multiplexing instead of the fork hack?==============================================================================8.2 TODO Autoimporting receivers.py from apps==============================================8.3 TODO docstrings====================8.4 TODO Minimize polling==========================Amazon charges for every call.  Less polling, lower invoice.  Someexponential backoff + out-of-band signal (view?) to wake up a runningreceiver process may be a good thing.8.5 TODO Custom exception to leave message in queue====================================================Provide a custom exception class that won't be handled by receiveloop (i.e. no backtrace) that can be used by receiver function toexplicitly leave message in queue without printing backtrace andalarming everyone.