Repository: neovim/neovim

Issue #21693: libnvim: in-process C API/ABI/FFI
Created: 2023-01-08T20:48:36Z
Problem
Nvim C API is not officially supported. (However, our cmake definition has a libnvim target, which is used by e.g. VimR)
Expected behavior

Decide which C types our C interface will use. Do we need to avoid structs?
Provide *.h headers for consumers include.
Declare SONAME
Document ABI (not API) contract. (+test coverage)
Document the correct usage of libnvim

Open questions

extra considerations for UIs that want to embed libnvim ? #2034

current process for UIs embedding Nvim in-process is: link to libnvim and speak msgpack to it (same as out-of-process UI).


via @dundargoc  :

libnvim. Does it need explicitly need to be a static library? Couldn't we just not set the type?
Apparently you're not supposed to specify if a library is meant to be static or shared anyway so outside projects can choose what type of library they want to use.



Related

https://flameeyes.blog/2012/03/07/why-foreign-function-interfaces-are-not-an-easy-answer/



example: moving from uint32_t to uint64_t for the parameters of a function: ... such a change in the C ABI would make the change incompatible, as the size of the parameters on the stack doubled.




https://flameeyes.blog/2010/10/08/linkers-and-names/



neither the link editor nor the loader have a real concept of “ABI version”, which is why they don’t enforce it at all

Duplicate issue #801: make new VimL implementation use only public API, but with C
Created: 2014-06-02T16:12:10Z
Rationale:


Why C and not remote protocol based on msgpack?
There are lots of things which require fast VimL execution. I am very unsure that one can implement fast vim.eval if it will first send message to NeoVim and then send message to VimL interpreter, which will in turn send NeoVim message back which NeoVim should resend to Python. I.e. it doubles amount of messages that are needed to be send.


Why only public API?

It can be abstracted so that at some point in the future one may choose whether to use embedded VimL interpreter or VimL interpreter in the separate process. Depending on implementation (details below) it may be useful for parallel execution of plugins or for providing virtual environments (e.g. to workaround plugin conflicts or to allow language-agnostic sandbox mode which restricts the API).
Current Python/Ruby/whatever interfaces are very incomplete: a large set of features is available only via vim.eval or even vim.command. Restricting VimL to public API is the easiest way to make sure that any feature that programmer may use appears in all languages almost at once via saner API then vim.eval/vim.command.
Currently there is a number of things which are not possible even with VimL: e.g. you cannot get a list of signs without using :redir hack. Restricting VimL to public API is an easy way to make sure that we cover all functions with more or less sane API. This will also allow us to move formatting code from C to lua.
If some restrictions (like thread safety) are put on functions they are most likely officially put only on public API functions. Unofficially thread safety should be a property of any or almost any function, but actually it may be implemented via some hacks at some intermediate level. There is some code for parallel VimL execution, this restriction should make it safer. It is also less stress for developers: to make something …-safe for all languages you need to make API …-safe, not make VimL and API safe.



Some details I am not sure about:


Where should be scopes kept? Keeping them in NeoVim will provide faster access, keeping them in VimL implementation seems cleaner. I would stick with the second.


Where should specifically be v: scope variables kept. Some variables (like v:count) are logical to keep inside NeoVim, some may be kept anywhere (like v:progname: it is readonly), some are logical to keep only in VimL interpreter (like v:key). I would special-case v: dictionary, making it dictionary-like object, in long term they may be optimized for direct API calls or interpreter state access (keeping dictionary-like object for compatibility with those people who use v: as a scope dictionary directly).


What to do with options like &isident: some options are needed for parsing/execution, but are checked really often to use API calls or are checked without using public API via non-VimL-interpreter vim functions. I would stick with what is in the next point:


I still use non-API Vim functions in the parser. To separate NeoVim and VimL interpreter this should be avoided. I think best is to move all functions I can to VimL without breaking NeoVim (only applicable for functions that were used for nothing, but implementing features moved to VimL), all functions I can’t should reside in files disconnected from NeoVim as much as possible, ideally in a separate namespace (directory) without imports not from that namespace (this is mostly relevant for memory.c).
This includes functions using options: they will accept options as an argument and their caller will be made responsible for providing correct ones.
&isident-depending options which appear to use optimized variant of option will require additional overhead for checking whether option changed, this check may be promoted to the higher level (e.g. happen only once per expression parsing or per one command parsing). If such functions appear in a separate namespace as they are used by NeoVim then it is supposed that they define optimized structure and option converter in that namespace and accept optimized structure in argument. Caching will be responsibility of the caller.

Closing comment (2023-03-08T17:41:18Z): Next steps for public C API are tracked in #21693

Where should be scopes kept?

Related: :help nvim_get_context()