arlolra/check — pull request #12 by Ryman (2013-09-09 02:22:15+00:00)
Title: Performance improvement for Dump (and tests)

Sets up an interval tree (one optimised for small integer values) with all port ranges specified in the policies. The query time is relative to the output size, and the interval tree can in theory also perform multiple queries in basically the same time. (If needed in future for multiple ip/port queries. Not currently implemented as it's not needed)

It gets a set of rules as the result of a port query, and iterates them to find matching rules. It's writing straight to the output buffer currently as that should cause less allocations. I can fix up some of the handlers to do similar once they're considered stable (if required).

Interestingly, the current bottleneck for dumping is checking IPs for matched rules. Could do a number of fun things to make that super fast too (the rules per port are actually static after each exit list rebuild, therefore the set of IPs they accept/reject is also static), but we're already dumping 900 results for a given up/port in less than 0.0002 seconds...

So we're likely bottlenecked by other things (I haven't profiled the handlers) if at all so I don't think it's worth pursuing unless just out of curiosity/perfectionism :) And I'm pretty sure IsTor (main site & Torbutton) is the main hit for the service, so we're already good there. I can't imagine the dump being that frequent?