MB-4894 Illegal state transition in warmup

Change-Id: I2d2cdbfb9a20ee34c01a484734c6515ab7a9a248
Reviewed-on: http://review.couchbase.org/13938
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
Refactor: move warmup related stats to a separate chunk

The time period where these stats are interesting is relatively
limited. There is no point of returning them every time a user
requests the stats. (in most cases they'll be ignored anyway)

Change-Id: I3160e4bdac287c2dc80769eeb56a2c72a5c95d1e
Reviewed-on: http://review.couchbase.org/13931
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-3466 Add bytes as an alias for mem_used

Change-Id: Ic7b506aea3e29b739b13d02aaec8a2149cd95d4e
Reviewed-on: http://review.couchbase.org/13714
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Remove TCMalloc from ep-engine

Ep-engine should use whatever allocator is being used by
memcached. As a result we should link TCMalloc into
memcached and not ep-engine. The only reason tcmalloc was
linked to ep-engine before was so that we could grab
stats from TCMalloc. This logic however will be moved into
memcached and ep-engine will be able to access it though
engine api's.

Change-Id: I8e5725abcd5eba20f3d77defec4b6068dfd50c01
Reviewed-on: http://review.couchbase.org/13776
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Added ability to track tcmalloc allocations

Change-Id: I36146728ef395bb15f34c324eb411874250b4c26
Reviewed-on: http://review.couchbase.org/13777
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Remove an unused member variable from TapCheckpointState.

Change-Id: I69d1cd21e5f4e43950597650945b479bcf678342
Reviewed-on: http://review.couchbase.org/13941
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Remove unnecessary copy constructor from TapCheckpointState.

Change-Id: I8a8cd4f4e90d6cd951834950cc2004bd6d88b57d
Reviewed-on: http://review.couchbase.org/13942
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Fix build break with Sun Studio compilers

The prototype for the allocator hook functions have a "C"
linkage and we try to set them up with functions with C++
linkage.. warning == errors --> build break

Change-Id: If6c8825e5b65a3081b97c30700dd02de684920f2
Reviewed-on: http://review.couchbase.org/13946
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Sergey Avseyev <sergey.avseyev@gmail.com>
Tested-by: Sergey Avseyev <sergey.avseyev@gmail.com>
Reviewed-by: Dustin Sallings <dustin@spy.net>
Fix compilation error

The build process generates generated_configuration.cc based on
configuration.json. The generated C file casted 4294967296 to
the size_t type which is not valid in most systems, where size_t
is signed and generally it has a size of 32 bits.

Change-Id: I89980ede4684c0d10fc474022ec122b657b77874
Reviewed-on: http://review.couchbase.org/13981
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
CouchKVStore to enable direct I/O to CouchDB files

CouchKVStore is a new backend of ep-engine that utilize couchstore api
to directly read and write to a CouchDB file

Change-Id: I8b2cd933fbf731d0bbe202d9c3c747db64106569
Reviewed-on: http://review.couchbase.org/13927
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Damien Katz <damien@couchbase.com>
Fix compilation error in CouchKVStore

Change-Id: I4fa789f3f8bcf71a615b880b9334c684a35ba403
Reviewed-on: http://review.couchbase.org/14018
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Jin Lim <jin@couchbase.com>
Revert "Fix compilation error in CouchKVStore"

This reverts commit 3ec7d2f08243d004a182884b1ec81c6b95c21dd9.

This changeset don't fix all of the build problems introduced in
the previous patch.. Need to back it out in order to back out the
other.

Change-Id: I07866e00151e6899fa6dc371ebf380e92e051928
Reviewed-on: http://review.couchbase.org/14019
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Revert "CouchKVStore to enable direct I/O to CouchDB files"

This reverts commit bacb148a77f11f8109b6e124eb6206f97f842749.

This changeset doesn't build on FreeBSD or Solaris

Change-Id: If1d3f0675d36ebe7c4f5cd52d16fa7ef19fcba76
Reviewed-on: http://review.couchbase.org/14015
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Track max seqno of deleted items per vbucket

In order to guarantee monotonically increasing seqno values even for
recreated items i.e., items that are deleted and reintroduced using
an add or a set command, we track the max seqno of all deleted items
seen so far. Then, when an item is reintroduced, we pick this max
seqno value, increment it by 1 and assign it to the "new" item.
Although conservative, this guarantees that if an item is thus
reintroduced, it gets a seqno that's higher than its previous value.

Change-Id: Ibb30fc5a90db21e05bf1043bfcb1c1050f3913ce
Load the per vbucket maxDeletedSeqno values during warmup

Change-Id: I5b5ca992d90f25328791ede70827efcefe319daf
delete-with-meta implementation in ep-engine

Change-Id: I879aa4f4d4a71b6080613673f6c72e7408d0ee4c
Introduce static consts to track StoredValue states

This is a refactoring change set and introduces a few static consts
to track the different states of the id attribute of a StoredValue
object.

Change-Id: I9224a55f1e025ec48e77762e79ea838c1fda3b98
Add new stored value states for tracking deleted items

For servicing getMetaData() requests for possibly deleted items, we
create a temporary item, add it to the hashtable and schedule a
background fetch of its metadata from the persistent store. After the
fetch completes, we restore its metadata in memory (if it was found
in the persistent store) or mark it as non existent. We also set it
to automatically expire after a predetermined but configurable amount
of time. As such, in order to correctly track the background fetch,
the following new stored value states are required:
1. deleted_key: after the fetch completes, if the item was found
   deleted in the persistent store, it is marked as deleted_key.
2. non_existent_key: after the fetch completes, if the item was not
   found in the persistent store, it is marked as having never
   existed. engine_key_noent will be returned if the item is queried.

This change set introduces these states. The actual logic of handling
deletes as described here is part of subsequent change sets.

Change-Id: I7511e24b6e3ae51f1ba5b06a92b0b83ec0a2b160
Add a new engine parameter for expiring temp items

This configurable parameter's value will be used to automatically
expire temporary items created in response to getMetaData() requests
for keys that aren't found in memory.

Change-Id: I5abb4d86c2832247682034426081b789cdf15f58
Add a resetValue() method to StoredValue class

Change-Id: I6b60a2629c82dfa96969e86a8233036a34c0b4d1