rackerlabs/yagi — pull request #14 by manalilatkar (2013-07-02 08:37:13+00:00)
Title: Rm216

This commit includes changes according to your comments:
1) I am not seeing how the cuf_handler is distinguishing by event_type. The serialization format seems to assume the format of the compute.instance.exists.* events, (for example requiring the audit_period_beginning / audit_period_ending items in the payload), but Yagi handles all types of notifications, each type will have different data. Instantaneous events (pretty much everything but .exists.* ) won't have the audit_period_* items, most of the compute.instance.* events from nova will have similar instance data, but image.* events from glance won't.

==True. Earlier, all this would have led to keyerrors. Now before passing the messages to cuf_handler, the __call__ method will filter out the messages whose event_type is there in config.filters.(This filter functionality is configurable and can be applied to any no. of events for any handler.) Hence the whole message traffic won't be passed to cuf_handler.

2)Yagi will be getting compute.instance.exists.verified.old, and sending that through AtomPub, but then also sending it in CUF format as compute.instance.exists.verified, so the CUF handler will need to change the event_type in it's output to AtomHopper. It will also need to change the id, since AtomHopper can check for duplicates, and throw an error. If the .verified, and .verified.old events have the same atom:id, and atomhoppers dup detection is enabled, one of them will throw an error.
==We did not know about the AtomHopper dup detection. Now the xml is created with a new uuid with exists.verified.Thus, AtomHopper will have 2 notifications- exists.verified and exists.verified.old with different uuids. Do we also want the uuid of the old message? We will have to change the serializer to accomodate 2 uuids.

3) In general, there seems to be alot of code duplication between the AtomPub and CUF handlers. There is already some code divergence, as there is some fixes to AtomPub protocol issues in AtomPub handler's handle_messages method to handle auth reconnect issues and HTTP 409 Conflict (due to atomhoppers dup detection) that CUF handler does not have in the same code. The duplication can be refactored later (properly factoring out some functionality in handle_messages so CUF handler could be a subclass of AtomPub handler, for example, will take some refactoring of AtomPub handler, and I can understand if you want to avoid touching existing functionality at the moment), but we will want to make sure those AtomPub handler bugfixes get into CUF handler as well.

==Yes, there seems to be a lot of code duplication for the 2 handlers. I have rebased the code with the 409 response handling and included it in the http connection and now both cuf_pub and atom_pub use the same http_connection object. We can still refactor the error response handling code present in the handle_messages method of both the handlers but I thought we can do that in a different commit.

4)Is deployment info different for both the handlers? Currently, we get it from config.[event_feed][atom_categories], but we might need to change that bit if the deployment info, i.e the data center and region will be different for both the handlers.