Fix RecordIn and RecordDrop stats

Summary:
1. fix possible overflow of the two stats by using uint64_t
2. use a similar source of data to calculate RecordDrop. Previous one is not correct.

Test Plan: See outputs of db_bench settings, and the results look reasonable

Reviewers: MarkCallaghan, ljin, igor

Reviewed By: igor

Subscribers: rven, leveldb, yhchiang, dhruba

Differential Revision: https://reviews.facebook.net/D28155
Add db_bench option --report_file_operations

Summary: Add db_bench option --report_file_operations

Test Plan:
./db_bench --report_file_operations
Observe outputs on # of file operations

Reviewers: ljin, MarkCallaghan, sdong

Reviewed By: sdong

Subscribers: yhchiang, rven, igor, dhruba

Differential Revision: https://reviews.facebook.net/D27945
suppress JDK8 errors for #385
Make sure WAL is synced for DB::Write() if write batch is empty

Summary: This patch makes it a contract that if an empty write batch is passed to DB::Write() and WriteOptions.sync = true, fsync is called to WAL.

Test Plan: A new unit test

Reviewers: ljin, rven, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, MarkCallaghan, leveldb

Differential Revision: https://reviews.facebook.net/D28365
Merge pull request #386 from EugenePig/java8

suppress JDK8 errors for #385
Improve DBTest.GroupCommitTest: artificially slowdown log writing to trigger group commit

Summary: In order to avoid random failure of DBTest.GroupCommitTest, artificially sleep 100 microseconds in each log writing.

Test Plan: Run the test in a machine where valgrind version of the test always fails multiple times and see it always succeed.

Reviewers: igor, yhchiang, rven, ljin

Reviewed By: ljin

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D28401
update HISOTRY.md for new release

Summary:
as title

Test Plan:
n/a
Turn -Wshadow back on

Summary: It turns out that -Wshadow has different rules for gcc than clang. Previous commit fixed clang. This commits fixes the rest of the warnings for gcc.

Test Plan: compiles

Reviewers: ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28131
Turn off -Wshadow

Summary:
So glibc is not -Wshadow-safe, so we need to turn it off :(

      error: ‘int sigaction(int, const sigaction*, sigaction*)’ hides
      constructor for ‘struct sigaction’

The rest of the changes in this diff is that we include .h files under rocksdb namespace, which is a no-no.

Test Plan: compiles now

Reviewers: ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28413
Turn on -Wshadow for travis
Make PartialCompactionFailure Test more robust.

Summary: Make PartialCompactionFailure Test more robust.

Test Plan:
export ROCKSDB_TESTS=PartialCompactionFailure
./db_test

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28425
[RocksJava] -WShadow improvements

Minor corrections to resolve -WShadow build problems with RocksJava code.
Merge pull request #387 from fyrz/RocksJava-WShadow-Fix

[RocksJava] -WShadow improvements
make DropWritesFlush deterministic

Summary:
TEST_WaitForFlush should wait until it sees error when parameter is set
to true so we don't need to loop and timeout

Test Plan: ROCKSDB_TESTS=DropWritesFlush ./db_test

Reviewers: sdong, igor

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28419
Make PartialCompactionFailure Test more robust again.

Summary:
Make PartialCompactionFailure Test more robust again by
blocking background compaction until we simulate the
file creation error.

Test Plan:
export ROCKSDB_TESTS=PartialCompactionFailure
./db_test

Reviewers: sdong, igor, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28431
Correctly test both compaction styles in CompactionDeletionTriggerReopen

Summary:
CompactionDeletionTriggerReopen wasn't actually testing universal
compaction.

Test Plan: db_test

Reviewers: sdong, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28443
Redesign pending_outputs_

Summary:
Here's a prototype of redesigning pending_outputs_. This way, we don't have to expose pending_outputs_ to other classes (CompactionJob, FlushJob, MemtableList). DBImpl takes care of it.

Still have to write some comments, but should be good enough to start the discussion.

Test Plan: make check, will also run stress test

Reviewers: ljin, sdong, rven, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28353
Fix compile
Fix implicit compare
Turn on -Wshadow

Summary:
Only one more try, I promise.

I talked to Jim and he mentioned that if we include our system includes with -isystem rather than with -I, that signals to the compile that those are system includes and thus no warnings are issued. So I turned our glibc includes into system includes and now we no longer get the warning from there, making us shadow-warning-free!

Test Plan: compiles with both clang and gcc

Reviewers: sdong, yhchiang, rven, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28479