HADOOP-8551. fs -mkdir creates parent directories without the -p option (Daryn Sharp and John George via bobby)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363435 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans via tgraves)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363454 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3673. libhdfs: fix some compiler warnings. Contributed by Colin Patrick McCabe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363457 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3675. libhdfs: follow documented return codes. Contributed by Colin Patrick McCabe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363459 13f79535-47bb-0310-9956-ffa450edef68
HDFS-1249. With fuse-dfs, chown which only has owner (or only group) argument fails with Input/output error. Contributed by Colin Patrick McCabe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363466 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3077. QJM main patch

Rev 6/28/2012:
- Move some logic back from AsyncLoggerSet into QJM
- improve comments, inline some methods which only had one call-site
- separate out timeout constants for different quorums
- add TODO for another test case
- Rejigger URL generation for synchronization into AsyncLogger interface,
- set NSInfo when creating the logger instead of taking it as a parameter to newEpoch()
- Add a test case to make sure the paxos behavior works during recovery (aborted recovery, then a new NN does recovery)
- Clean up logs to use shorter debug strings for protobufs

Rev 7/16/2012:
- Journal side changes after changes to HDFS-3190 (renamed PersistentLong, etc)
- remove TODO about synchronizing GetJournalEditsServet: already done
- Rename edits servlet to getJournal, remove assocated TODO. Also remove httpsPort variable, since its unused now that SPNEGO is in place
- Limit the amount of queued data in a logger
- Add journalnode to bin/hdfs
- Some initial test code to try to exhaustively/randomly test
- Move from using RemoteEditLogProto to SegmentStateProto, including an md5hash field. TODO: hook up the md5 verification!
- javadoc in AsyncLogger
- reorganize AsyncLoggerSet function order, add more javadoc
- add a close() call to IPCLoggerChannel. TODO: add a test
- add Storage.getSingularStorageDir() per atm review
- rename curSegmentTxId to lastSegmentTxId in recovery response
- Rename getEpochInfo to getJournalState, move httpPort fetching there, where it makes more sense
- Remove a TODO in QJournalProtocolPB that was addressed
- Rename everything into client/server packages
- Fix an issue with ExitException termination
- Add patch to ExitUtil so that the exit-tracking state can be reset

Rev 7/17/2012:
- Fix a few unit tests that got broken by earlier moving of stuff around (trivial)
- Fix findbugs warnings, add excludes where necessary
- Add confs for all timeouts
- AsyncLoggerSet.close() should close underlying logger IPCs
- Lots more javadoc in QJournalProtocol, JNStorage, Journal
- Remove fancy attempt at synchronization for syncLog for now. Will put it back later if necessary for performance.

Rev 7/19/2012:
- Fix bin/hdfs script - elif vs if typo in previous rev
- Remove an empty format() stub in FSEditLog (added in a previous rev, but unused)
- revert a spurious FileSystem change which should not be in 3077
- Add TODO about close() hanging when remote side is down (noticed while manual testing)
- improve TODO AssertionError for empty logs to include the path of the empty log
- Add a TODO about null segments during recovery, a case which occurred during manual testing
Implement purgeLogsOlderThan(...) by remotely RPCing to the JNs.
Also purges the old paxos decisions.
Add an INFO log for purgeEditLogs
Fix Journal to read the storage info when it is first initialized, even before newEpoch() is called.

This is necessary when starting NNs in standby state: the edits transfer
servlet tries to verify the storage info, but if there's no active writer,
it was previously empty (nsid=0).
Fix getEditLogManifest to fetch httpPort if necessary

This is necessary when the SBN starts up - it hasn't called getJournalInfo
on its own, so this is needed to init the httpPort

FileJournalManager should sort its list of edit logs -- JournalNode calls FJM.getRemoteEditLogs directly
Genericize format(), make initializeSharedEdits() format non-file shared edits
Fix bug where RemoteEditLogManifest has gaps in the case of QJM. TODO: add regression test for this