Caching & IOStrategy @ SwayDB
Tags: java, scala, kotlin, storage
Published: 2020-03-27 04:30:41

---
title: Caching & IOStrategy @ SwayDB
published: true
description: Controlling cache and IO concurrency in SwayDB
tags: java, scala, kotlin, storage
---

___We all know that Memory IO is 50-200 times faster than Disk IO!___

__Caching__ plays a decent role in boosting __read__ and __compaction__ performance but some machines & smaller devices (eg: mobile phones) might have not have enough memory for cache so a __configurable cache__ allowing __disabled__, __partial__ or __full__ cache is required.

__Note__ - A `Segment(.seg)` file in SwayDB is simply a byte array that stores other bytes arrays like keys, values, indexes etc (`Array<Array<Byte>>`). All these bytes can be cached based on __any condition__ which is configurable. 

## Configuring IO and Cache
When accessing __any file__ with a custom format we generally 
- Open the file (`OpenResource`)
- Read the file header or info (`ReadDataOverview`) to understand the files content eg: format etc. 
- Finally read the content of the file (`Compressed` or `Uncompressed` data). 

The following sample `ioStrategy` function does exactly that where we get an `IOAction` that describes what IO is being performed by SwayDB and in our function we define how we want to perform IO (`IOStrategy`) for that action and __also configure caching__ for the read data/bytes. 

```java
.ioStrategy(  
  (IOAction ioAction) -> {  
    if (ioAction.isOpenResource()) {  
      //Here we are just opening the file so do synchronised IO because
      //blocking when opening a file might be cheaper than thread 
      //context switching. Also set cacheOnAccess to true so that other 
      //concurrent threads accessing the same file channel do not 
      //open multiple channels to the same file.
      return new IOStrategy.SynchronisedIO(true);  
  } else if (ioAction.isReadDataOverview()) {  
      //Data overview is always small and less than 128 bytes and can be 
      //read sychronously to avoid switching threads. Also cache
      //this data (cacheOnAccess) for the benifit of other threads and to save IO.
      return new IOStrategy.SynchronisedIO(true);  
  } else {  
      //Here we are reading actual content of the file which can be compressed
	  //or uncompressed. 
      IOAction.DataAction action = (IOAction.DataAction) ioAction;  
      if (action.isCompressed()) {
        //If the data is compressed we do not want multiple threads to concurrently
        //decompress it so perform either Async or Sync IO for decompression
        //and then cache the compressed data. You can also read the compressed 
        //and decompressed size with the following code
        //IOAction.ReadCompressedData dataAction = (IOAction.ReadCompressedData) action;  
        //dataAction.compressedSize();  
        //dataAction.decompressedSize();
        return new IOStrategy.AsyncIO(true);  
      } else {  
        //Else the data is not compressed so we allow concurrent access to it. 
        //Here cacheOnAccess can also be set to true but that could allow multiple
        //threads to concurrently cache the same data. If cacheOnAccess is required 
        //then use Asyc or Sync IO instead.
        return new IOStrategy.ConcurrentIO(false);  
      }  
    }  
  }  
)
```

You will find the above `ioStrategy` property in __all data-blocks__ that form a Segment - [SortedKeyIndex](https://swaydb.simer.au/configurations/sortedKeyIndex/?language=java/), [RandomKeyIndex](https://swaydb.simer.au/configurations/randomKeyIndex/?language=java/), [BinarySearchIndex](https://swaydb.simer.au/configurations/binarySearchIndex/?language=java/), [MightContainIndex](https://swaydb.simer.au/configurations/mightContainKeyIndex/?language=java/) & [ValuesConfig](https://swaydb.simer.au/configurations/valuesConfig/?language=java/).

A Segment itself is also a __data-block__ and it's `ioStrategy` can also be configured via [SegmentConfig](https://swaydb.simer.au/configurations/segmentConfig/?language=java/).

## Cache control/limit with MemoryCache

___Caching should be controlled so that it does not lead to memory overflow!___

 You can __enable__ or __disable__ caching for any or all of the following
- Bytes within a Segment  (`ByteCacheOnly`).
- Parsed key-values (`KeyValueCacheOnly`).
- Or all the above (`MemoryCache.All`).

By default `ByteCacheOnly` is used because `KeyValueCacheOnly` uses an in-memory `SkipList` and inserts to a large `SkipList` are expensive which is not useful for general use-case.  But `KeyValueCacheOnly`  can be useful for applications that perform multiple reads to the same data and if that data rarely changes. 

An `Actor` configuration is also required here which manages the cache in the background. You can configure the Actor to be a `Basic`, `Timer` or `TimerLoop`.

The following demoes how to configured all caches. 

```java
//Byte cache only
.setMemoryCache(  
  MemoryCache  
    .byteCacheOnlyBuilder()
    .minIOSeekSize(4096)
    .skipBlockCacheSeekSize(StorageUnits.mb(4))  
    .cacheCapacity(StorageUnits.gb(2))  
    .actorConfig(new ActorConfig.Basic((ExecutionContext) DefaultConfigs.sweeperEC()))  
)
//or key-value cache only
.setMemoryCache(  
  MemoryCache  
    .keyValueCacheOnlyBuilder()  
    .cacheCapacity(StorageUnits.gb(3))  
    .maxCachedKeyValueCountPerSegment(Optional.of(100))  
    .actorConfig(new Some(new ActorConfig.Basic((ExecutionContext) DefaultConfigs.sweeperEC())))  
)
//or enable both the above.
.setMemoryCache(  
  MemoryCache  
    .allBuilder()  
    .minIOSeekSize(4096)  
    .skipBlockCacheSeekSize(StorageUnits.mb(4))  
    .cacheCapacity(StorageUnits.gb(1))  
    .maxCachedKeyValueCountPerSegment(Optional.of(100))  
    .sweepCachedKeyValues(true)  
    .actorConfig(new ActorConfig.Basic((ExecutionContext) DefaultConfigs.sweeperEC()))  
)
```

### `minIOSeekSize`
The [blockSize]([https://en.wikipedia.org/wiki/Block_(data_storage)](https://en.wikipedia.org/wiki/Block_(data_storage))) which set the minimum number of bytes to read for each IO. For example in the above configuration if you ask for `6000 bytes` then `4096 * 2 bytes` will be read. 

The value to set depends on your machines block size. On Mac this can be read with the following [command]([https://apple.stackexchange.com/questions/78802/what-are-the-sector-sizes-on-mac-os-x](https://apple.stackexchange.com/questions/78802/what-are-the-sector-sizes-on-mac-os-x)):
```
diskutil info / | grep "Block Size"
```
which returns
> Device Block Size: 4096 Bytes
> Allocation Block Size: 4096 Bytes

### `skipBlockCacheSeekSize`
This skips the `BlockCache` and perform direct IO if the data size is greater than this value. 

### `cacheCapacity`
Sets the total memory capacity. On overflow the oldest data in the cache is dropped by the `Actor`. 

### `maxCachedKeyValueCountPerSegment`
If set, each `Segment` is initialised with a dedicated `LimitSkipList`. This cache is managed by the `Actor` or by the `Segment` itself if it gets deleted or when the max limit is reached.

### `sweepCachedKeyValues`
Enables clearing cached key-values via the `Actor`. If `false`, key-values are kept in-memory indefinitely unless the `Segment` gets deleted. This configuration can be used for smaller databases (eg: application configs) that read the same data more often. 

## Memory-mapping (MMAP)

MMAP can also be optionally enabled for all files. 
```java
Map<Integer, String, Void> map =  
  MapConfig  
    .functionsOff(Paths.get("myMap"), intSerializer(), stringSerializer())  
    .setMmapAppendix(true) //enable MMAP for appendix files
    .setMmapMaps(true)  //enable MMAP for LevelZero write-ahead log files
    .setSegmentConfig(  //configuring MMAP for Segment files
      SegmentConfig  
        .builder()  
         ...
         //either disable memory-mapping Segments
        .mmap(MMAP.disabled()) 
         //or enable for writes and reads.
        .mmap(MMAP.writeAndRead())  
         //or enable for reads only. 
        .mmap(MMAP.readOnly())  
        ...
     )  
    .get();  
  
map.put(1, "one");  
map.get(1); //Optional[one]
```

## Summary

You are in full control of __Caching__ & __IO__ and can configure it to suit your application needs. If your `IOStrategy` configurations uses only `AsyncIO` and `ConcurrentIO` then you can truely build __reactive applications__ which are non-blocking end-to-end other than the file system IO performed by `java.nio.*` classes. Support for [Libio](https://pagure.io/libaio) to provide aysnc file system IO can be implemented as a feature if requested.

## Useful links

-   [SwayDB](https://github.com/simerplaha/SwayDB)  on GitHub.
-   [Java](https://github.com/simerplaha/SwayDB.java.examples)  examples repo.
-   [Kotlin](https://github.com/simerplaha/SwayDB.kotlin.examples)  examples repo.
-   [Scala](https://github.com/simerplaha/SwayDB.scala.examples)  examples repo.
-   [Documentation](https://swaydb.simer.au/).