db_bench: sanity check on compression ratio

Summary: as requested by mark

Test Plan: make release

Reviewers: sdong, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19221
Fix some entries in HISTORY.md

Summary:
Add one entry to HISTORY.md that is failed to be added.
Move two other ones to the right location.

Test Plan: Only document

Reviewers: haobo, ljin

Subscribers: dhruba, yhchiang, leveldb

Differential Revision: https://reviews.facebook.net/D19233
[RocksDB] history change for 3.2

Summary: as title

Test Plan: none

Reviewers: ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19239
db_bench: measure the real latency of write/delete

Summary: as title

Test Plan: make release

Reviewers: haobo, sdong, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19227
[RocksDB] allow LDB tool to have customized key formatter

Summary: Currently ldb tool dump keys either in ascii format or hex format - neither is ideal if the key has a binary structure and is not readable in ascii. This diff also allows LDB tool to be customized in ways beyond DB options.

Test Plan: verify that key formatter works with some simple db with binary key.

Reviewers: sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19209
Update README.md

typo
Fix a rapidjson compile error in mac.

Summary:
This diff fixes the following compilation error in mac.
  ./third-party/rapidjson/reader.h:422:31: error: comparison of constant 256 with expression of type 'Ch' (aka 'char') is always true
        [-Werror,-Wtautological-constant-out-of-range-compare]
                                  if ((sizeof(Ch) == 1 || e < 256) && escape[(unsigned char)e])
                                                          ~ ^ ~~~
  1 error generated.

Test Plan: make db_test

Reviewers: haobo, sdong, igor, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19245
Fix compile error caused in LDB tool

Summary:
Fixed the following compile error.
    tools/reduce_levels_test.cc:89:31: error: no matching function for call to 'InitFromCmdLineArgs'
      LDBCommand* level_reducer = LDBCommand::InitFromCmdLineArgs(args);
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ./util/ldb_cmd.h:56:22: note: candidate function not viable: requires 3 arguments, but 1 was provided
      static LDBCommand* InitFromCmdLineArgs(
                         ^
    ./util/ldb_cmd.h:62:22: note: candidate function not viable: requires 4 arguments, but 1 was provided
      static LDBCommand* InitFromCmdLineArgs(
                         ^
    1 error generated.

Test Plan:
make reduce_levels_test
./reduce_levels_test

Reviewers: haobo, ljin, sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19251
Fix compile error in reduce_levels_test.

Summary:
Fixed the following compile error.
    tools/reduce_levels_test.cc:89:31: error: no matching function for call to 'InitFromCmdLineArgs'
      LDBCommand* level_reducer = LDBCommand::InitFromCmdLineArgs(args);
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ./util/ldb_cmd.h:56:22: note: candidate function not viable: requires 3 arguments, but 1 was provided
      static LDBCommand* InitFromCmdLineArgs(
                         ^
    ./util/ldb_cmd.h:62:22: note: candidate function not viable: requires 4 arguments, but 1 was provided
      static LDBCommand* InitFromCmdLineArgs(
                         ^
    1 error generated.

Test Plan:
make reduce_levels_test
./reduce_levels_test

Reviewers: haobo, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19251
Revert the default setting of InitFromCmdLineArgs().

Summary:
Revert the default setting of InitFromCmdLineArgs() as all the callers
currently provide full set of arguments.

Test Plan:
make reduce_levels_test
./reduce_levels_test

Reviewers: haobo, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19257
added a dot
escaped the special characters and added a dot
Merge pull request #181 from nanwu/master

Corrected the output message in mac-install-gflags.sh
[Java] Enable compression_ratio option in DbBenchmark.java

Summary:
Enable the random values in Java DB Bench to be generated based
on the compression_ratio specified in the command-line arguments.

Test Plan:
make rocksdbjava
java/jdb_bench.sh

Reviewers: sdong, ankgup87, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19101
Improve an assertion in RandomGenerator::Generate() in db_bench.

Summary:
RandomGenerator::Generate() currently has an assertion len < data_.size().
However, it is actually fine to have len == data_.size().
This diff change the assertion to len <= data_.size().

Test Plan:
make db_bench
./db_bench

Reviewers: haobo, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19269
Allow compaction to reclaim storage more effectively.

Summary:
This diff allows compaction to reclaim storage more effectively.
In the current design, compactions are mainly triggered based on
the file sizes.  However, since deletion entries does not have
value, files which have many deletion entries are less likely
to be compacted.  As a result, it may took a while to make
deletion entries to be compacted.

This diff address issue by compensating the size of deletion
entries during compaction process: the size of each deletion
entry in the compaction process is augmented by 2x average
value size.  The diff applies to both leveled and universal
compacitons.

Test Plan:
develop CompactionDeletionTrigger
make db_test
./db_test

Reviewers: haobo, igor, ljin, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19029
Reorder the member variables of FileMetaData to improve cache locality.

Summary:
Move stats related member variables of FileMetaData to the bottom to
improve cache locality of normal DB operations.

Test Plan: make

Reviewers: haobo, ljin, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19287
Fixed heap-buffer-overflow issue when Options.num_levels > 7.

Summary:
Currently, when num_levels has been changed to > 7, internally
it will not resize max_bytes_for_level_multiplier_additional.
As a result, max_bytes_for_level_multiplier_additional.size() will
be smaller than num_levels, which causes heap-buffer-overflow.

Test Plan: make all check

Reviewers: haobo, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19275
Remove MemTableRep::GetIterator(const Slice& slice)

Summary: It seems to me that when ever function MemTableRep::GetIterator(const Slice& slice) is used, we can use MemTableRep::GetDynamicPrefixIterator() instead. Just delete it to simplify the codes.

Test Plan: make all check

Reviewers: yhchiang, ljin

Reviewed By: ljin

Subscribers: xjin, dhruba, haobo, leveldb

Differential Revision: https://reviews.facebook.net/D19281
Fixed a comparison between signed and unsigned integers in options.cc

Summary:
Fixed the following warning:

util/options.cc: In constructor ‘rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(const rocksdb::Options&)’:
util/options.cc:157:58: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   if (max_bytes_for_level_multiplier_additional.size() < num_levels) {
                                                             ^

Test Plan: make all check

Reviewers: haobo, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19293