HADOOP-8681. add support for HTTPS to the web UIs. (tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3758. TestFuseDFS test failing. Contributed by Colin Patrick McCabe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371555 13f79535-47bb-0310-9956-ffa450edef68
Revert HDFS-3719. See discussion there and HDFS-3770 for more info.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372544 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3789. JournalManager#format() should be able to throw IOException. Contributed by Ivan Kelly.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372566 13f79535-47bb-0310-9956-ffa450edef68
Amend HADOOP-8659. Native libraries must build with soft-float ABI for Oracle JVM on ARM.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372583 13f79535-47bb-0310-9956-ffa450edef68
Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372605 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3276. initializeSharedEdits should have a -nonInteractive flag. Contributed by Todd Lipcon.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372628 13f79535-47bb-0310-9956-ffa450edef68
Merge trunk into branch


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1372630 13f79535-47bb-0310-9956-ffa450edef68
HDFS-3796. Allow EditLogFileOutputStream to skip fsync() in tests
HDFS-3765. initializeSharedEdits (with some fixes to pass tests by todd)

Dont allow InitializeSharedEdits to run when no shared edits are configured

Another fix for initializeSharedEdits to work when checkpoints are not on segment boundaries

Add debug logging to initializeSharedEdits, fix a bug where an extra END_LOG_SEGMENT would get inserted

A few small NN side fixes so initializeSharedEdits works with QJM
HDFS-3793. Implement format() for QJM

Add QJM change for format()
Wire QJM.format() rpc
log message when formatting
Throw JournalNotFormattedException when using unformatted journal
Fix QJM.format() to wait for responses and throw exception on failure
should call checkFormatted() when requesting edit log manifest
format() API should throw IOE
Hook up format() RPC in JournalNodeRpcServer
Implement hasSomeData() for QJM
Stop auto-formatting on first access (unit tests probably broken now)
Fix testMismatchedNNIsRejected now that format isnt automatic
Fix TestQJMWithFaults for non-autoformatting
Fix TestJournal for non auto-format
Add format call for TestEpochsAreUnique
Add javadoc and annotation to JournalNotFormattedException
HDFS-3795. Validate journal dir at startup
HDFS-3798. Avoid throwing NPE if finalizeLogSegment() is called on an invalid segment
deal with empty files in recovery path

Fixes around crashes right at the start of a segment
Make journal() call take segmentTxId as parameter

This addresses an issue where a server misses both the
finalize() of one segment and the start() of the next segment,
causing it to "merge" the two segments into a longer one.
This causes problems later with recovery.

This could be addressed on the client side by having it
notice the miss of startLogSegment() and stop journaling
at that point until the next one. However, it is useful
to have on the server side as an extra sanity-check
nonetheless.
Add some test infrastructure to allow tests to be more deterministic (not in use yet)

Use a deterministic order for TestQJM

use sameThreadExec
Improvements to fault test:

- unwrap exceptions when they're thrown by the reflection-based proxy
- don't assume that we will recover exactly to the last acked seqno + 1. Instead, we should recover to *at least* that

Add second randomized fault test
IPCLoggerChannel: set httpPort if it misses getJournalState IPC

TODO: this is kidn of hacky
Maintain a writer epoch, fix recovery protocol so finalized logs always win out

TODO: add unit tests for recovery code
Beginnings of a targeted test towards one of the faults fixed last weekend

Need to actually get this test to fail without the bug fixes: TODO revert the bug fixes, make this test fail