GitHub PullRequestReviewCommentEvent at 2014-06-23 18:47:21
Repository: basho_docs
Actor: coderoshi
Payload: {"action":"created","comment":{"url":"https://api.github.com/repos/basho/basho_docs/pulls/comments/14092523","id":14092523,"diff_hunk":"@@ -15,522 +15,753 @@ moved: {\n }\n ---\n \n-## Overview\n+[Bitcask](https://github.com/basho/bitcask) is an Erlang application\n+that provides an API for storing and retrieving key/value data using\n+log-structured hash tables that provide very fast access. The\n+[design](http://downloads.basho.com/papers/bitcask-intro.pdf) of Bitcask\n+was inspired, in part, by log-structured filesystems and log file\n+merging.\n \n-[Bitcask](https://github.com/basho/bitcask) is an Erlang application that\n-provides an API for storing and retrieving key/value data into a log-structured\n-hash table that provides very fast access. The\n-[design](http://downloads.basho.com/papers/bitcask-intro.pdf) of Bitcash owes a lot to the\n-principles found in log-structured file systems and draws inspiration from a\n-number of designs that involve log file merging.\n-\n-### Strengths:\n+## Bitcask's Strengths\n \n * **Low latency per item read or written**\n \n-    This is due to the write-once, append-only nature of the Bitcask database\n-    files.\n+    This is due to the write-once, append-only nature of Bitcask\n+    database files.\n \n-* **High throughput, especially when writing an incoming stream of random items**\n+* **High throughput, especially when writing an incoming stream of\n+    random items**\n \n-    Because the data being written doesn't need to be ordered on disk and\n-    because the log structured design allows for minimal disk head movement\n-    during writes, these operations generally saturate the I/O and disk\n-    bandwidth.\n+    Write operations to Bitcask generally saturate I/O and disk\n+    bandwidth for two reasons: because (1) data that is written to \n+    Bitcask doesn't need to be ordered on disk, and (2) the\n+    log-structured design of Bitcask allows for minimal disk head \n+    movement during writes.\n \n * **Ability to handle datasets larger than RAM without degradation**\n \n-    Because access to data in Bitcask involves direct lookup from an\n-    in-memory hash   table finding data on disk is very efficient, even when\n-    data sets are very large.\n+    Access to data in Bitcask involves direct lookup from an in-memory\n+    hash table. This makes finding data very efficient, even when data\n+    setes are very large.\n \n * **Single seek to retrieve any value**\n \n-    Bitcask's in-memory hash-table of keys point directly to locations on disk\n-    where the data lives. Bitcask never uses more than one disk seek to read a\n-    value and sometimes, due to file-system caching done by the operating\n-    system, even that isn't necessary.\n+    Bitcask's in-memory hash table of keys points directly to locations\n+    on disk where the data lives. Bitcask never uses more than one disk\n+    seek to read a value and sometimes even that isn't necessary due to\n+    filesystem caching done by the operating system.\n \n * **Predictable lookup _and_ insert performance**\n \n-    As you might expect from the description above, read operations have a\n-    fixed, predictable behavior. What you might not expect is that this is\n-    also true for writes. Write operations are at most a seek to the end of\n-    the current file open writing and an append to that file.\n+    For the reasons listed above, read operations from Bitcask have\n+    fixed, predictable behavior. This is also true of writes to Bitask\n+    because write operations require, at most, one seek to the end of\n+    the current open file followed by and append to that file.\n \n * **Fast, bounded crash recovery**\n \n-    Due to the append-only write once nature of Bitcask files, recovery is easy\n-    and fast. The only items that might be lost are partially written records\n-    at the tail of the file last opened for writes. Recovery need only review\n-    the last record or two written and verify CRC data to ensure that the data\n-    is consistent.\n+    Crash recovery is easy and fast with Bitcask because Bitcask files\n+    are append only and write once. The only items that may be lost are\n+    partially written records at the tail of the last file that was\n+    opened for writes. Recovery operations need to review only the last\n+    record or two written and verify CRC data to ensure that the data is\n+    consistent.\n \n * **Easy Backup**\n \n-    In most systems backup can be very complicated but here again Bitcask\n-    simplifies this process due to it's append-only write once disk format.\n-    Any utility that archives or copies files in disk-block order will properly\n-    backup or copy a Bitcask database.\n+    In most systems, backup can be very complicated. Bitcask simplifies\n+    this process due to its append-only, write-once disk format. Any\n+    utility that archives or copies files in disk-block order will\n+    properly back up or copy a Bitcask database.\n \n-### Weaknesses:\n+### Weaknesses\n \n * Keys must fit in memory\n \n-    Bitcask keeps all keys in memory at all times, this means that your system\n-    must have enough memory to contain your entire keyspace with room for other\n-    operational components and operating system resident filesystem buffer\n-    space.\n+    Bitcask keeps all keys in memory at all times, which means that your\n+    system must have enough memory to contain your entire keyspace, plus\n+    additional space for other operational components and operating-\n+    system-resident filesystem buffer space.\n \n ## Installing Bitcask\n \n-Riak ships with Bitcask included within the distribution. In fact it is the\n-default storage engine and no separate installation is required.\n+Bitcask is the default storage engine for Riak. You can verify that\n+Bitcask is currently being used as the storage backend with the\n+`[[riak|riak Command Line]]` command interface:\n \n-The default configuration values found in the `app.config` for Bitcask are as\n-follows:\n+```bash\n+riak config effective | grep backend\n+```\n \n-```erlang\n+If this operation returns anything other than `bitcask`, read\n+the [[next section|Bitcask#enabling-bitcask]] for instructions on\n+switching the backend to Bitcask.\n+\n+## Enabling Bitcask\n+\n+You can set Bitcask as the storage engine using each node's\n+[[configuration files]]:\n+\n+```riakconf\n+storage_backend = bitcask\n+```\n+\n+```appconfig\n+{riak_kv, [\n+\t{storage_backend, riak_kv_bitcask_backend},\n+\t%% Other riak_kv settings...\n+]},\n+%% Other settings...\n+```\n+\n+## Configuring Bitcask\n+\n+Bitcask enables you to configure a wide variety of its behaviors, from\n+object storage syncing strategy to merge settings and more.\n+\n+The default configuration values for Bitcask are as follows:\n+\n+```riakconf\n+bitcask.data_root = ./data/bitcask\n+bitcask.io_mode = erlang\n+```\n+\n+```appconfig\n %% Bitcask Config\n     {bitcask, [\n         {data_root, \"/var/lib/riak/bitcask\"}\n     ]},\n ```\n \n-## Configuring Bitcask\n-\n-Modify the default behavior by adding these settings into the `bitcask` section\n-in your [[app.config|Configuration Files]].\n+All other available settings can be added to your configuration files.\n \n ### Open Timeout\n \n-The `open_timeout` setting specifies the maximum time Bitcask will block on\n-startup while attempting to create or open the data directory. The value is\n-in seconds and the default is `4`. You generally need not change this value.\n-If for some reason the timeout is exceeded on open you'll see a log message\n-of the form: `\"Failed to start bitcask backend: ...`. Only then should you\n-consider a longer timeout.\n+The open timeout setting specifies the maximum time Bitcask will block\n+on startup while attempting to create or open the Bitcask data\n+directory. The default is 4 seconds.\n \n-```erlang\n+In general, you will not need to adjust this setting. If, however, you\n+begin to receive log messages of the form `Failed to start bitcask\n+backend: ...`, you may want to consider using a longer timeout.\n+\n+In the newer, `riak.conf`-based configuration system, open timeout is\n+specified using the `bitcask.sync.open_timeout` parameter, and can be\n+set in terms of seconds, minutes, hours, etc. The following example\n+sets the parameter to 10 seconds:\n+\n+```riakconf\n+bitcask.sync.open_timeout = 10s\n+```\n+\n+In the older, `app.config`-based configuration system, open timeout is\n+specified using the `open_timeout` parameter, and is set in seconds. The\n+following example sets the parameter to 10 seconds:\n+\n+```appconfig\n {bitcask, [\n-        ...,\n-            {open_timeout, 4} %% Wait time to open a keydir (in seconds)\n-        ...\n+    %% Other Bitcask settings...\n+        {open_timeout, 10},\n+    %% Other Bitcask settings...\n ]}\n ```\n \n ### Sync Strategy\n \n-The `sync_strategy` setting changes the durability of writes by specifying\n-when to synchronize data to disk. The default setting protects against data\n-loss in the event of application failure (process death) but leaves open a\n-small window wherein data could be lost in the event of complete system\n-failure (e.g. hardware, O/S, power).\n+Bitcask enables you to configure the durability of writes by specifying\n+when to synchronize data to disk, i.e. by choosing a sync strategy. The\n+default setting (`none`) writes data into operating system buffers that\n+will be written to disk when those buffers are flushed by the operating\n+system. If the system fails before those buffers are flushed, e.g. due\n+to power loss, that data is lost.\n \n-The default mode, `none`, writes data into operating system buffers which\n-which will be written to the disks when those buffers are flushed by the\n-operating system. If the system fails (power loss, crash, etc.) before\n-before those buffers are flushed to stable storage that data is lost.\n+Thus, using the default setting of `none` protects against data loss in the\n+event of application failure, i.e. process death, but leaves open a\n+small window in which data could be lost in the event of a complete\n+system failure, e.g. hardware or OS failure.\n \n-This is prevented by the setting `o_sync` which forces the operating system\n-to flush to stable storage at every write. The effect of flushing each\n-write is better durability, however write throughput will suffer as each\n-write will have to wait for the write to complete.\n+This possibility can be prevented by choosing the `o_sync` sync\n+strategy, which forces the operating system to flush to stable storage\n+at write time for every write. The effect of flushing each write is\n+better durability, although it should be noted that write throughput\n+will suffer because each write will have to wait for the write to\n+complete.\n \n-#### Available Sync Strategies\n+The following sync strategies are available:\n \n   * `none` --- lets the operating system manage syncing writes (default)\n-  * `o_sync` --- uses the `O_SYNC` flag which forces syncs on every write\n-  * `{seconds, N}` --- Riak will force Bitcask to sync every `N` seconds\n+  * `o_sync` --- uses the `O_SYNC` flag, which forces syncs on every write\n+  * Time interval --- Riak will force Bitcask to sync at specified intervals\n \n-```erlang\n+The following are possible configurations:\n+\n+\n+```riakconf\n+bitcask.sync.strategy = none\n+bitcask.sync.strategy = o_sync\n+bitcask.sync.interval = 10s\n+```\n+\n+```appconfig\n {bitcask, [\n     ...,\n-        {sync_strategy, none}, %% Let the O/S decide when to flush to disk\n+        {sync_strategy, none},\n+        {sync_strategy, o_sync},\n+        {sync_strategy, {seconds, 10}}, %% The time interval must be specified in seconds\n     ...\n ]}\n ```\n \n-<div class=\"note\">\n-<div class=\"title\">Bitcask doesn't actually set <tt>O_SYNC</tt> on\n-Linux</div>\n-At the time of this writing, due to an unresolved Linux <a\n-href=\"http://permalink.gmane.org/gmane.linux.kernel/1123952\">kernel issue</a>\n-related to the <a\n-href=\"https://github.com/torvalds/linux/blob/master/fs/fcntl.c#L146..L198\">implementation\n-of <tt>fcntl</tt></a>, it turns out that Bitcask will not set the <tt>O_SYNC</tt> flag on the file opened for writing. The call to\n-<tt>fcntl</tt> doesn't fail. Instead, it is silently ignored by the Linux kernel.\n-You will notice a <a\n-href=\"https://github.com/basho/riak_kv/commit/6a29591ecd9da73e27223a1a55acd80c21d4d17f#src/riak_kv_bitcask_backend.erl\">warning\n-message</a> in the log files of the format:<br /><tt>{sync_strategy,o_sync} not\n-implemented on Linux</tt><br /> indicating that this issue exists on your system.\n-Without the <tt>O_SYNC</tt> setting enabled there is the potential for data\n-loss if the OS or system dies (power outtage, kernel panic, reboot without a\n-sync, etc.) with dirty buffers not yet written to stable storage.\n-</div>\n+#### `O_SYNC` on Linux\n+\n+At the moment, Bitcask doesn't acutally set `O_SYNC` on Linux, due to an\n+unresolved [kernel issue](http://permalink.gmane.org/gmane.linux.kernel/1123952)\n+related to implementation of `[fcntl](https://github.com/torvalds/linux/blob/master/fs/fcntl.c#L146..L198)`.\n+\n+When a file is opened for writing by Bitcask, it will not set the `O_SYNC`\n+flag. The call to `fcntl` doesn't fail, but rather silently ignored by\n+the Linux kernel. You may notice [warning messages](https://github.com/basho/riak_kv/commit/6a29591ecd9da73e27223a1a55acd80c21d4d17f#src/riak_kv_bitcask_backend.erl)\n+of the following format in your log files:\n+\n+```log\n+{sync_strategy, o_sync} not implemented on Linux`\n+```\n \n-### Disk-Usage and Merging Settings\n-\n-Riak KV stores each vnode partition of the ring as a separate Bitcask directory within the configured bitcask data directory. Each of these\n-directories will contain multiple files with key/value data, one or more\n-\"hint\" files that record where the various keys exist within the data files,\n-and a write lock file. The design of Bitcask allows for recovery even when\n-data isn't fully synchronized to disk (partial writes). This is accomplished\n-by maintaining data files that are append-only (never modified in-place) and\n-are never reopened for modification (only reading).\n-\n-The data management strategy trades disk space for operational efficiency.\n-There can be a significant storage overhead that is un-related to your\n-working data set but can be tuned to best fit your usage. In short, disk\n-space is used until a threshold is met, then unused space is reclaimed\n-through a process of merging. The merge process traverses data files and\n-reclaims space by eliminating out-of-date versions of key/value pairs writing\n-only the current key/value pairs to a new set of files within the directory.\n-\n-The merge process is affected by the settings described below. In the\n-discussion, \"dead\" refers to keys that are no longer the latest value or\n-those that have been deleted; \"live\" refers to keys that are the newest value\n-and have not been deleted.\n+These messages indicate that this issue exists on your system. Without\n+the `O_SYNC` setting enabled, there is the potential for data loss if\n+the OS or system dies, e.g. due to a power outage, kernel panic, reboot\n+without sync, etc., with dirty buffers not yet written to stable\n+storage.\n+\n+### Disk Usage and Merging Settings\n+\n+Riak stores each [[vnode partition|Riak Glossary#vnodes]] of the\n+[[ring|Clusters#the-ring]] as a separate Bitcask directory within the\n+configured Bitcask data directory.\n+\n+Each of these directories will contain multiple files with key/value\n+data, one or more \"hint\" files that record where the various keys exist\n+within the data files, and a write lock file. The design of Bitcask\n+allows for recovery even when data isn't fully synchronized to disk\n+(partial writes). This is accomplished by maintaining data files that\n+are append-only (i.e. never modified in-place) and are never reopened\n+for modification (i.e. only for reading).\n+\n+This data management strategy trades disk space for operational\n+efficiency. There can be a significant storage overhead that is\n+unrelated to your working data set but can be tuned in a way that best\n+fits your use case. In short, disk space is used until a threshold is\n+met, while unused space is reclaimed through a process of merging. The\n+merge process traverses data files and reclaims space by eliminating\n+out-of-date versions of key/value pairs, writing only the current \n+key/value pairs to a new set of files within the directory.\n+\n+The merge process is affected by all of the settings described in the\n+sections below. In those sections, \"dead\" refers to keys that no longer\n+contain the most up-to-date values, while \"live\" refers to keys that\n+do contain the most up-to-date value and have not been deleted.\n \n #### Max File Size\n \n The `max_file_size` setting describes the maximum permitted size for any\n single data file in the Bitcask directory. If a write causes the current\n-file to exceed this size threshold then that file is closed, and a new file\n-is opened for writes.\n+file to exceed this size threshold then that file is closed, and a new\n+file is opened for writes. The default is 2 GB.\n \n Increasing `max_file_size` will cause Bitcask to create fewer, larger\n-files, which are merged less frequently while decreasing it will cause\n-Bitcask to create more numerous, smaller files, which are merged more\n-frequently. If your ring size is 16 your servers could see as much as 32GB\n-of data in the bitcask directories before the first merge is triggered\n-irrespective of your working set size. Plan storage accordingly and don't\n-be surprised by larger than working set on disk data sizes.\n+files that are merged less frequently, while decreasing it will cause\n+Bitcask to create more numerous, smaller files that are merged more\n+frequently.\n+\n+To give an example, if your ring size is 16, your servers could see as\n+much as 32GB of data in the bitcask directories before the first merge\n+is triggered, irrespective of your working set size. You should plan\n+storage accordingly and be aware that it is possible to see disk data\n+sizes that are larger than the working set.\n+\n+In the newer, `riak.conf`-based configuration system, `max_file_size`\n+can be specified using kilobytes, megabytes, etc. The following example\n+sets the max file size to 1 GB:\n+\n+```riakconf\n+bitcask.max_file_size = 1GB\n+```\n \n-The default is `16#80000000`, which is 2GB in bytes.\n+In the older, `app.config`-based configuration system, `max_file_size`\n+must be specified in bytes. The default of 2 GB, for example, would be\n+set as `16#80000000`:\n \n-```erlang\n+```appconfig\n {bitcask, [\n-        ...,\n-        {max_file_size, 16#80000000}, %% 2GB default\n-        ...\n+    ...,\n+    {max_file_size, 16#80000000}, %% 2GB default\n+    ...\n ]}\n ```\n \n #### Hint File CRC Check\n \n-During startup Bitcask will read from `.hint` files to build its in memory\n-representation of the key space, falling back to the `.data` files if\n-necessary.  This reduces the amount of data that must be read from the\n-disk during startup, and thereby the time required to startup.  The \n-`require_hint_crc` setting will determine if a `.hint` file will be used\n-when it does not contain a CRC signature, but is otherwise able\n-to be read properly.  \n+During startup, Bitcask will read from `.hint` files in order to build\n+its in-memory representation of the key space, falling back to `.data`\n+files if necessary. This reduces the amount of data that must be read\n+from the disk during startup, thereby also reducing the time required to\n+start up. You can configure Bitcask to either disregard `.hint` files\n+containing a CRC value or to use them anyway.\n \n-* `true` --- disregard any `.hint` file that does not contain a CRC value \n-* `false` ---  use `.hint` files that are otherwise valid except for missing CRC\n+If you are using the newer, `riak.conf`-based configuration system,\n+you can instruct Bitcask to disregard `.hint` files that do not contain\n+a CRC value by setting the `hintfile_checksums` setting to `strict` (the\n+default):\n \n-```erlang\n+```riakconf\n+bitcask.hintfile_checksums = strict\n+```\n+\n+To use Bitcask in a backward-compatible mode that allows for `.hint`\n+files without CRC signatures, change this setting to `allow_missing`.\n+\n+If you are using the older, `app.config`-based configuration system,\n+you can instruct Bitcask to disregard `.hint` files that do not contain\n+a CRC value by setting `require_hint_crc` to `true`, as in this example:\n+\n+```appconfig\n {bitcask, [\n-        ...,\n-        {require_hint_crc, true},\n-        ...\n+    ...,\n+    {require_hint_crc, true},\n+    ...\n ]}\n ```\n \n-#### IO Mode\n+Setting `require_hint_crc` to `false` will instruct Bitcask to accept\n+otherwise-valid `.hint` files without CRC values.\n+\n+#### I/O Mode\n \n The `io_mode` setting specifies which code module Bitcask should use for \n-file access.  Valid settings are:\n+file access. The available settings are:\n \n-* `erlang` (default) use Erlang file module\n-* `nif` use native implemented function module written in C\n+* `erlang` (default) --- Writes are made via Erlang's built-in file API\n+* `nif` --- Writes are made via direct calls to the POSIX C API\n \n-```erlang\n+The following example sets `io_mode` to `erlang`:\n+\n+```riakconf\n+bitcask.io_mode = erlang\n+```\n+\n+```appconfig\n {bitcask, [\n-        ...,\n-        {io_mode, erlang},\n-        ...\n+    ...,\n+    {io_mode, erlang},\n+    ...\n ]}\n ```\n \n-#### Merge Window\n+In general, the `nif` IO mode provies higher throughput for certain\n+workloads, but it has the potential to negatively impact the Erlang VM,\n+leading to higher worst-case latencies and possible throughput collapse.\n \n-The `merge_window` setting lets you specify when during the day merge\n-operations are allowed to be triggered. Valid options are:\n+#### Merge Policy\n \n-* `always` --- no restrictions (default)\n-* `never` --- merge will never be attempted\n-* `{Start, End}` --- hours during which merging is permitted, where `Start` and\n-  `End` are integers between 0 and 23. A window defined as `{1, 3}` means that\n-  merging will be allowed to start between 01:00 and 03:59.\n+Bitcask enables you to select a merge policy, i.e. when during the day\n+merge operations are allowed to be triggered. The valid options are:\n \n-If merging has a significant impact on performance of your cluster, or your\n-cluster has quiet periods in which little storage activity occurs, you may\n-want to change this setting from the default.\n+* `always` --- No restrictions on when merge operations can occur (default)\n+* `never` --- Merge will never be attempted\n+* Start and end time --- Hours during which merging is permitted\n \n-A common way to limit the impact of merging is to create separate merge windows \n-for each node in the cluster and ensure that these windows do not overlap. This\n-ensures that at most one node at the time can be affected by merging, leaving \n-the remaining nodes to handle requests. The main drawback of this approach is that\n-merges will occur less frequently, leading to increased disk space usage.\n+If you are using the newer, `riak.conf`-based configuration system, you\n+can select a merge policy using the `merge.policy` setting. The\n+following example sets the merge policy to `never`:\n \n-The default is `always`.\n+```riakconf\n+bitcask.merge_policy = never\n+``` \n \n-```erlang\n+If you opt to specify start and end hours for merge operations, you can\n+do so by specifying the `merge.window.start` and `merge.window.end`\n+settings. Each setting is an integer between 0 and 23, with 0 signifying\n+midnight and 23 specifying 11 pm. The following example enables merging\n+between 3 am and 4:59 pm:\n+\n+```riakconf\n+bitcask.merge.window.start = 3\n+bitcask.merge.window.end = 17\n+```\n+\n+If you are using the older, `app.config`-based configuration system, you\n+can specify a merge policy using the `merge_window` setting. The \n+following example sets the merge policy to `never`:\n+\n+```appconfig\n {bitcask, [\n     ...,\n-        {merge_window, always}, %% Span of hours during which merge is acceptable.\n+    {merge_window, never},\n     ...\n ]}\n ```\n \n-<div class=\"note\"><div class=\"title\"> `merge_window` and Multi-Backend</div>\n-When using Bitcask with [[Multi-Backend|Multi]], please note that if you\n-wish to use a merge window, you *must* set it in the global `bitcask`\n-section of your `app.config`.  `merge_window` settings in per-backend\n-sections are ignored.\n+If you wish to specify a time interval instead, you can select a window\n+as an Erlang tuple of the form `{Start, End}`, where `Start` and `End`\n+are both integers between 0 and 23, with 0 signifying midnight and 23\n+signifying 11 pm. The following example enables merging between 3 am and\n+4:59 pm:\n+\n+```appconfig\n+{bitcask, [\n+    ...,\n+    {merge_window, {3, 17}},\n+    ...\n+]}\n+```\n+\n+<div class=\"note\">\n+<div class=\"title\"><code>merge_window</code> and the Multi backend</div>\n+If you are using the older configuration system and using Bitcask with\n+the [[Multi]] backend, please note that if you wish to use a merge\n+window, you <em>must</em> set it in the global <code>bitcask</code>\n+section of your configuration file. <code>merge_window</code> settings\n+in per-backend sections are ignored.\n </div>\n \n+If merging has a significant impact on performance of your cluster, or\n+if your cluster has quiet periods in which little storage activity\n+occurs, you may want to change this setting from the default.\n+\n+A common way to limit the impact of merging is to create separate merge\n+windows  for each node in the cluster and ensure that these windows do\n+not overlap. This ensures that at most one node at a time can be\n+affected by merging, leaving the remaining nodes to handle requests.\n+The main drawback of this approach is that merges will occur less\n+frequently, leading to increased disk space usage.\n \n #### Merge Triggers\n \n-Merge triggers determine under what conditions merging will be\n-invoked.\n+Merge triggers determine the conditions under which merging will be\n+invoked. These conditions fall under two basic categories:\n \n-* Fragmentation --- The `frag_merge_trigger` setting describes what ratio of\n-  dead keys to total keys in a file will trigger merging. The value of this\n-  setting is a percentage (0-100). For example, if a data file contains 6\n-  dead keys and 4 live keys, then merge will be triggered at the default\n-  setting. Increasing this value will cause merging to occur less often,\n+* **Fragmentation** --- This describes the ratio of dead keys to total\n+  keys in a file that will trigger merging. The value of this setting is\n+  an integer percentage (0-100). For example, if a data file contains 6\n+  dead keys and 4 live keys, merge will be triggered the default setting\n+  (60%). Increasing this value will cause merging to occur less often,\n   whereas decreasing the value will cause merging to happen more often.\n \n-  Default is: `60`\n-\n-* Dead Bytes --- The `dead_bytes_merge_trigger` setting describes how much\n-  data stored for dead keys in a single file will trigger merging. The\n-  value is in bytes. If a file meets or exceeds the trigger value for dead\n-  bytes, merge will be triggered. Increasing the value will cause merging\n-  to occur less often, whereas decreasing the value will cause merging to\n-  happen more often.\n+* **Dead Bytes** --- This setting describes how much data stored for dead\n+  keys in a single file will trigger merging. If a file meets or exceeds\n+  the trigger value for dead bytes, merge will be triggered. Increasing\n+  the value will cause merging to occur less often, whereas decreasing\n+  the value will cause merging to happen more often.\n \n   When either of these constraints are met by any file in the directory,\n-  Bitcask will attempt to merge files.\n+  Bitcask will attempt to merge files. The default is 512 MB.\n+\n+If you are using the newer, `riak.conf`-based configuration system,\n+you can set the triggers described above using `merge.triggers.fragmentation`\n+and `merge.triggers.dead_bytes`, respectively. The former is expressed\n+as an integer between 0 and 100, whereas the latter can be expressed in\n+terms of kilobytes, megabytes, gigabytes, etc. The following example\n+sets the dead bytes threshold to 55% and the fragmentation threshold to\n+1 GB:\n+\n+```riakconf\n+bitcask.merge.fragmentation = 55\n+bitcask.merge.triggers.dead_bytes = 1GB\n+```\n \n-The default is `536870912`, which is 512MB in bytes.\n+If you are using the older, `app.config`-based configuration system, you\n+can set the triggers described above using the `dead_bytes_merge_trigger`\n+and `frag_merge_trigger` parameters, respectively. The former is\n+expressed as an integer between 0 and 100, whereas the latter can be\n+expressed in terms of bytes. The following example sets the dead bytes\n+threshold to 55% and the fragmentation threshold to 1 GB:\n \n-```erlang\n+```appconfig\n {bitcask, [\n-        ...,\n-        %% Trigger a merge if any of the following are true:\n-            {frag_merge_trigger, 60}, %% fragmentation >= 60%\n-        {dead_bytes_merge_trigger, 536870912}, %% dead bytes > 512 MB\n-        ...\n+    ...,\n+    {frag_merge_trigger, 60},\n+    {dead_bytes_merge_trigger, 1073741824},\n+    ...\n ]}\n ```\n \n #### Merge Thresholds\n \n-Merge thresholds determine which files will be chosen to be included in a\n-merge operation.\n-\n-- _Fragmentation_: The `frag_threshold` setting describes what ratio of\n-    dead keys to total keys in a file will cause it to be included in the\n-    merge. The value of this setting is a percentage (0-100). For example,\n-    if a data file contains 4 dead keys and 6 live keys, it will be included\n-    in the merge at the default ratio. Increasing the value will cause fewer\n-    files to be merged, decreasing the value will cause more files to be\n-    merged.\n-\n-    Default is: `40`\n-\n-- _Dead Bytes_: The `dead_bytes_threshold` setting describes the minimum\n-    amount of data occupied by dead keys in a file to cause it to be included\n-    in the merge. Increasing the value will cause fewer files to be merged,\n-    decreasing the value will cause more files to be merged.\n-\n-    Default is: `134217728` which is 128MB in bytes\n-\n-- _Small File_: The `small_file_threshold` setting describes the minimum\n-    size a file must have to be _excluded_ from the merge. Files smaller\n-    than the threshold will be included. Increasing the value will cause\n-    _more_ files to be merged, decreasing the value will cause _fewer_ files\n-    to be merged.\n+Merge thresholds determine which files will be chosen for inclusion in\n+a merge operation.\n+\n+* **Fragmentation** --- This setting describes which ration of dead keys\n+  to total keys in a file will cause it to be included in the merge. The\n+  value of this setting is a percentage (0-100). For example, if a data\n+  file contains 4 dead keys and 6 live keys, it will be included in the\n+  merge at the default ratio (40%). Increasing the value will cause\n+  fewer files to be merged, while decreasing the value will cause more\n+  files to be merged.\n+\n+* **Dead Bytes** --- This setting describes which ratio the minimum\n+  amount of data occupied by dead keys in a file to cause it to be\n+  included in the merge. Increasing this value will cause fewer files to\n+  be merged, while decreasing this value will cause more files to be\n+  merged. The default is 128 MB.\n+\n+* **Small File** --- This setting describes the minimum size a file must\n+  be to be _excluded_ from the merge. Files smaller than the threshold\n+  will be included. Increasing the value will cause more files to be\n+  merged, while decreasing the value will case fewer files to be merged.\n+  The default is 10 MB.\n+\n+If you are using the newer, `riak.conf`-based configuration system, you\n+can set the thresholds described above using the `merge.thresholds.fragmentation`,\n+`merge.thresholds.dead_bytes`, and `merge.threshold.small_file` settings,\n+respectively.\n+\n+The `fragmentation` setting is expressed as an integer\n+between 0 and 100, and the `dead_bytes` and `small_file` settings can be\n+expressed in terms of kilobytes, megabytes, gigabytes, etc. The\n+following example sets the fragmentation threshold to 45%, the\n+dead bytes threshold to 200 MB, and the small file threshold to 25 MB:\n+\n+```riakconf\n+bitcask.merge.thresholds.fragmentation = 45\n+bitcask.merge.thresholds.dead_bytes = 200MB\n+bitcask.merge.thresholds.small_file = 25MB\n+```\n \n-    Default is: `10485760` while is 10MB in bytes\n+If you are using the older, `app.config`-based configuration system,\n+you can set the thresholds described above using the `frag_threshold`,\n+`dead_bytes_threshold`, and `small_file_threshold` parameters,\n+respectively.\n \n-When any of these constraints are met for a single file, it will be\n-included in the merge operation.\n+The `frag_threshold` setting is expressed as an integer\n+between 0 and 100, while the `dead_bytes_threshold` and `small_file_threshold`\n+parameters are expressed in terms of bytes. The following example sets\n+the fragmentation threshold to 45%, the dead bytes threshold to 200 MB,\n+and the small file threshold to 25 MB:\n \n ```erlang","path":"source/languages/en/riak/ops/advanced/backends/bitcask.md","position":753,"original_position":753,"commit_id":"9db5003350d1d86459be0cfc515992b0d5f6ccd2","original_commit_id":"9db5003350d1d86459be0cfc515992b0d5f6ccd2","user":{"login":"coderoshi","id":63749,"avatar_url":"https://avatars.githubusercontent.com/u/63749?","gravatar_id":"707e3965f1630286a3dcaad501a10fed","url":"https://api.github.com/users/coderoshi","html_url":"https://github.com/coderoshi","followers_url":"https://api.github.com/users/coderoshi/followers","following_url":"https://api.github.com/users/coderoshi/following{/other_user}","gists_url":"https://api.github.com/users/coderoshi/gists{/gist_id}","starred_url":"https://api.github.com/users/coderoshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/coderoshi/subscriptions","organizations_url":"https://api.github.com/users/coderoshi/orgs","repos_url":"https://api.github.com/users/coderoshi/repos","events_url":"https://api.github.com/users/coderoshi/events{/privacy}","received_events_url":"https://api.github.com/users/coderoshi/received_events","type":"User","site_admin":false},"body":"appconfig","created_at":"2014-06-23T18:47:21Z","updated_at":"2014-06-23T18:47:21Z","html_url":"https://github.com/basho/basho_docs/pull/972#discussion_r14092523","pull_request_url":"https://api.github.com/repos/basho/basho_docs/pulls/972","_links":{"self":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/comments/14092523"},"html":{"href":"https://github.com/basho/basho_docs/pull/972#discussion_r14092523"},"pull_request":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/972"}}},"pull_request":{"url":"https://api.github.com/repos/basho/basho_docs/pulls/972","id":14191723,"html_url":"https://github.com/basho/basho_docs/pull/972","diff_url":"https://github.com/basho/basho_docs/pull/972.diff","patch_url":"https://github.com/basho/basho_docs/pull/972.patch","issue_url":"https://api.github.com/repos/basho/basho_docs/issues/972","number":972,"state":"open","title":"Ready: Update backend-specific docs","user":{"login":"lucperkins","id":1523104,"avatar_url":"https://avatars.githubusercontent.com/u/1523104?","gravatar_id":"af481d7274821037b1a807995b4a6598","url":"https://api.github.com/users/lucperkins","html_url":"https://github.com/lucperkins","followers_url":"https://api.github.com/users/lucperkins/followers","following_url":"https://api.github.com/users/lucperkins/following{/other_user}","gists_url":"https://api.github.com/users/lucperkins/gists{/gist_id}","starred_url":"https://api.github.com/users/lucperkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lucperkins/subscriptions","organizations_url":"https://api.github.com/users/lucperkins/orgs","repos_url":"https://api.github.com/users/lucperkins/repos","events_url":"https://api.github.com/users/lucperkins/events{/privacy}","received_events_url":"https://api.github.com/users/lucperkins/received_events","type":"User","site_admin":false},"body":"- [x] Bitcask\r\n  - [x] Disk reclamation settings from issue #1057 \r\n- [x] Memory\r\n - [x] Ensure that `max_memory` is documented as a *per vnode* setting, not per node or per cluster\r\n- [x] Multi\r\n\r\nLevelDB changes are being made in a different branch/PR.","created_at":"2014-03-31T19:03:05Z","updated_at":"2014-06-23T18:47:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":"04c20396580989b49d670cd865a87eea07b13043","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/basho/basho_docs/pulls/972/commits","review_comments_url":"https://api.github.com/repos/basho/basho_docs/pulls/972/comments","review_comment_url":"https://api.github.com/repos/basho/basho_docs/pulls/comments/{number}","comments_url":"https://api.github.com/repos/basho/basho_docs/issues/972/comments","statuses_url":"https://api.github.com/repos/basho/basho_docs/statuses/9db5003350d1d86459be0cfc515992b0d5f6ccd2","head":{"label":"basho:features/lp/update-backends-docs","ref":"features/lp/update-backends-docs","sha":"9db5003350d1d86459be0cfc515992b0d5f6ccd2","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?","gravatar_id":"ce5141b78d2fe237e8bfba49d6aff405","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":2401217,"name":"basho_docs","full_name":"basho/basho_docs","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?","gravatar_id":"ce5141b78d2fe237e8bfba49d6aff405","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/basho_docs","description":"Basho Products Documentation","fork":false,"url":"https://api.github.com/repos/basho/basho_docs","forks_url":"https://api.github.com/repos/basho/basho_docs/forks","keys_url":"https://api.github.com/repos/basho/basho_docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/basho_docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/basho_docs/teams","hooks_url":"https://api.github.com/repos/basho/basho_docs/hooks","issue_events_url":"https://api.github.com/repos/basho/basho_docs/issues/events{/number}","events_url":"https://api.github.com/repos/basho/basho_docs/events","assignees_url":"https://api.github.com/repos/basho/basho_docs/assignees{/user}","branches_url":"https://api.github.com/repos/basho/basho_docs/branches{/branch}","tags_url":"https://api.github.com/repos/basho/basho_docs/tags","blobs_url":"https://api.github.com/repos/basho/basho_docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/basho_docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/basho_docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/basho_docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/basho_docs/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/basho_docs/languages","stargazers_url":"https://api.github.com/repos/basho/basho_docs/stargazers","contributors_url":"https://api.github.com/repos/basho/basho_docs/contributors","subscribers_url":"https://api.github.com/repos/basho/basho_docs/subscribers","subscription_url":"https://api.github.com/repos/basho/basho_docs/subscription","commits_url":"https://api.github.com/repos/basho/basho_docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/basho_docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/basho_docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/basho_docs/issues/comments/{number}","contents_url":"https://api.github.com/repos/basho/basho_docs/contents/{+path}","compare_url":"https://api.github.com/repos/basho/basho_docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/basho_docs/merges","archive_url":"https://api.github.com/repos/basho/basho_docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/basho_docs/downloads","issues_url":"https://api.github.com/repos/basho/basho_docs/issues{/number}","pulls_url":"https://api.github.com/repos/basho/basho_docs/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/basho_docs/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/basho_docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/basho_docs/labels{/name}","releases_url":"https://api.github.com/repos/basho/basho_docs/releases{/id}","created_at":"2011-09-16T18:47:02Z","updated_at":"2014-06-23T17:27:16Z","pushed_at":"2014-06-23T17:27:15Z","git_url":"git://github.com/basho/basho_docs.git","ssh_url":"git@github.com:basho/basho_docs.git","clone_url":"https://github.com/basho/basho_docs.git","svn_url":"https://github.com/basho/basho_docs","homepage":"http://docs.basho.com","size":42614,"stargazers_count":48,"watchers_count":48,"language":"CSS","has_issues":true,"has_downloads":true,"has_wiki":true,"forks_count":71,"mirror_url":null,"open_issues_count":131,"forks":71,"open_issues":131,"watchers":48,"default_branch":"master"}},"base":{"label":"basho:2.0.0","ref":"2.0.0","sha":"006d0cbfdb6170e678ecdf2a02c73f735e186544","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?","gravatar_id":"ce5141b78d2fe237e8bfba49d6aff405","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":2401217,"name":"basho_docs","full_name":"basho/basho_docs","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?","gravatar_id":"ce5141b78d2fe237e8bfba49d6aff405","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/basho_docs","description":"Basho Products Documentation","fork":false,"url":"https://api.github.com/repos/basho/basho_docs","forks_url":"https://api.github.com/repos/basho/basho_docs/forks","keys_url":"https://api.github.com/repos/basho/basho_docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/basho_docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/basho_docs/teams","hooks_url":"https://api.github.com/repos/basho/basho_docs/hooks","issue_events_url":"https://api.github.com/repos/basho/basho_docs/issues/events{/number}","events_url":"https://api.github.com/repos/basho/basho_docs/events","assignees_url":"https://api.github.com/repos/basho/basho_docs/assignees{/user}","branches_url":"https://api.github.com/repos/basho/basho_docs/branches{/branch}","tags_url":"https://api.github.com/repos/basho/basho_docs/tags","blobs_url":"https://api.github.com/repos/basho/basho_docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/basho_docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/basho_docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/basho_docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/basho_docs/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/basho_docs/languages","stargazers_url":"https://api.github.com/repos/basho/basho_docs/stargazers","contributors_url":"https://api.github.com/repos/basho/basho_docs/contributors","subscribers_url":"https://api.github.com/repos/basho/basho_docs/subscribers","subscription_url":"https://api.github.com/repos/basho/basho_docs/subscription","commits_url":"https://api.github.com/repos/basho/basho_docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/basho_docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/basho_docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/basho_docs/issues/comments/{number}","contents_url":"https://api.github.com/repos/basho/basho_docs/contents/{+path}","compare_url":"https://api.github.com/repos/basho/basho_docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/basho_docs/merges","archive_url":"https://api.github.com/repos/basho/basho_docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/basho_docs/downloads","issues_url":"https://api.github.com/repos/basho/basho_docs/issues{/number}","pulls_url":"https://api.github.com/repos/basho/basho_docs/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/basho_docs/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/basho_docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/basho_docs/labels{/name}","releases_url":"https://api.github.com/repos/basho/basho_docs/releases{/id}","created_at":"2011-09-16T18:47:02Z","updated_at":"2014-06-23T17:27:16Z","pushed_at":"2014-06-23T17:27:15Z","git_url":"git://github.com/basho/basho_docs.git","ssh_url":"git@github.com:basho/basho_docs.git","clone_url":"https://github.com/basho/basho_docs.git","svn_url":"https://github.com/basho/basho_docs","homepage":"http://docs.basho.com","size":42614,"stargazers_count":48,"watchers_count":48,"language":"CSS","has_issues":true,"has_downloads":true,"has_wiki":true,"forks_count":71,"mirror_url":null,"open_issues_count":131,"forks":71,"open_issues":131,"watchers":48,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/972"},"html":{"href":"https://github.com/basho/basho_docs/pull/972"},"issue":{"href":"https://api.github.com/repos/basho/basho_docs/issues/972"},"comments":{"href":"https://api.github.com/repos/basho/basho_docs/issues/972/comments"},"review_comments":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/972/comments"},"review_comment":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/comments/{number}"},"commits":{"href":"https://api.github.com/repos/basho/basho_docs/pulls/972/commits"},"statuses":{"href":"https://api.github.com/repos/basho/basho_docs/statuses/9db5003350d1d86459be0cfc515992b0d5f6ccd2"}}}}