cjdelisle/cjdns — pull request #631 by Arceliar (2014-08-15 18:08:03+00:00)
Title: Ugly and unorganized optimization patch

Seriously not proud of some of the code in here, but I thought I should open the PR for review/comments.

Does the following:

1. Reorder priorities in the Janitor. Focus on learning more about nodes already in the store.
2. Also in the Janitor, do some level of per-link maintenance (select random node, select a link coming from the node, ping the child via that link). Do this instead of searching for new nodes 75% of the time (but never instead of searching for new links to existing nodes, or following up on something the search or nodestore asks us to check out).
3. In the nodestore, start trying to keep sensable values for link->linkState. Attempts to store what we think is the reach of the route from the link->parent to the link->child via the link. Uses this information in guessReachOfChild to make a slightly more educated guess.
4. If we find a bad route (brokenPath or a timeout), the nodestore queues up a ping to the parent of the last node, and to the last node again in the case of a timeout. If the path happens to be the one going through all bestParents (which it should be in most cases), then this should usually track down where the problem is by walking backwards towards the selfNode. In the case of a timeout, if there is a link legitimately down somewhere (not just a random drop due to someone's wifi or something), then it should eventually push reach low enough to evict the node from the store.
5. Incredibly minor point, changed the maximum number of search attempts before giving up on a search from 100 to 8. In most cases, we search for a nonexistent address, and we're just trying to find a valid next hop. At 2 searches per 30 seconds, and assuming 1 search attempt has about the same network overhead as 1 ping or getPeers request, this saves us the equivalent of 368 pings per minute (bringing the total down to at most 92, 60 from pings and 32 from search attempts, for an 80% reduction in idle traffic). The choice of 8 was arbitrary, I just figured that bringing the search traffic in line with half of the ping traffic seemed about in the right ballpark. I haven't noticed any negative side effects from doing this.

Side-effect is that the nodestore fills up more slowly (though the nodes it needs to know about for forwarding to work are there within a few seconds on my end), but the routes it ends up with seem to be shorter and more stable according to pingAll.js, at least for me.

Much of the code (including a big chunk in the janitor) wasn't really thought out before writing, so it may be worth re-writing before merging if you decide to keep any of this.