lalithsuresh/Scaling-HDFS-NameNode — pull request #8 by lalithsuresh (2011-12-17 19:01:47+00:00)
Title: Reduce the number of DB accesses during block/triplet handling

Guy, please have a look through this patch. I'll merge when I get three +1's.

The two commits include the following changes:

- Change BlockInfo class to remove the triplets representation completely, and only require a storage of the Datanode identifiers in the DB. This means that we no longer maintain a triplet representation as a linked list, and we no longer need the next/previous block ID columns in the TripletsTable either. This saves us *several* DB accesses.
- Added some helper methods in BlocksHelper to account for the above, and removed some which weren't needed anymore.
- Re-implement the BlockIterator to pull in all blocks associated with a Datanode as an ArrayList, and iterate over it with indices. The block report handling in reportDiff() was edited to match the changes and it now processes blocks correctly.
- Some other cosmetic changes to clean up things.

This doesn't break any previously passing unit test as far as I checked.