Repository: neovim
Issue: API: stream interface
Labels: enhancement,performance,api,refactor,needs:design,channels-rpc,complexity:high

I recently came up with the idea which may speed up API functions: in place of obtaining and returning some intermediate type like Object make them all work with some kind of streams. Specifically all API function signatures should look like

``` C
void func(APIOutStream out, APIInStream in);
```

where

``` C
typedef struct {
  bool (*push_dictionary_start)(void *cookie, size_t len);
  // …
  // basically the current msgpack API: first push container start with length, then values
  // values are pushed by one call, except for strings: sometimes it is more convenient to
  // push them in parts to avoid unnecessary allocations.
  bool (*push_string_start)(void *cookie, size_t len);
  bool (*push_string_part)(void *cookie, size_t len, char *data);
  bool (*push_string)(void *cookie, size_t len, char *data);
  bool (*error_out)(void *cookie, Error err);
  bool (*errorred)(void *cookie);
  void *cookie;
} APIOutStream;

typedef struct {
  ObjectType (*next_object_type)(void *cookie, Error *err);
  bool (*pull_dictionary_start)(void *cookie, size_t *ret_len, Error *err);
  // …
  bool (*pull_string)(void *cookie, size_t *ret_len, const char **ret_string, Error *err);
  void *cookie;
} APIInStream;
```

APIOutStream structure is supposed to be used to
1. Write msgpack values directly to the channel (which is OK as long as it is uses RBuffer).
2. Directly construct VimL values (#4934).
3. Directly construct lua values (#4411).

APIInStream is based on the same technique I previously discussed somewhere (AFAIR, gitter) with @tarruda: in place of pulling in the whole msgpack value, allocating a bunch of memory in process, pull this as needed, filling values placed on stack (and, e.g. `pull_string` may say that “returned string is usable as long as next stream function is not called”; to avoid alloctations though it may make sense to also pull string by parts, but unlike out stream this is going to be convenient almost never). So e.g. API function “id” (takes one argument and returns it) will look like (assuming recursion is OK)

``` C
#define WRITE(out, func, ...) \
  do { \
    APIOutStream out_ = (out); \
    if (!(out_.func(out_.cookie, __VA_ARGS__))) { \
      return; \
    } \
  } while (0)
#define READ(out, in, func, err, ...) \
  do { \
    APIInStream in_ = (in); \
    APIOutStream out_ = (out); \
    if (!(in_.func(in_.cookie, __VA_ARGS__, &err))) { \
      out_.error_out(out_.cookie, err); \
      return; \
    } \
  } while (0)
void api_id(APIOutStream out, APIInStream in)
  API(Object, (Object))
{
  Error err;
  switch (in.next_object_type(in.cookie, &err)) {
    case kObjectTypeMissing: {
      out.error_out(out.cookie, err);
      return;
    }
    case kObjectTypeNil: {
      if (!in.pull_nil(in.cookie, &err)) {
        assert(false);
      }
      if (!out.push_nil(out.cookie)) {
        return;
      }
      break;
    }
    case kObjectTypeString: {
      size_t len;
      const char *string;
      READ(out, in, pull_string, err, &len, &string));
      WRITE(out, push_string, len, sring);
      break;
    }
    // …
    case kObjectTypeDictionary: {
      size_t len;
      READ(out, in, pull_dictionary_start, err, &len));
      WRITE(out, push_dictionary_start, len);
      for (size_t i = 0; i < len * 2; i++) {  // len * 2: key + value
        api_id(out, in);
        if (out.errorred(out.cookie)) {
          return;
        }
      }
      break;
    }
  }
}
```

(This example was created in order to demonstrate how API is used, most real API functions will not have that much code size bump, especially given that there will be helpers for typical operations like “get VimL value out of in stream”.)