Question: Best practice for allowing multiple Vulkan instances as part of 'renderer' objects

I'm writing a library in C to abstract multiple graphics APIs including Vulkan behind the same interface.
Currently, the library allows for the creation of multiple 'renderer' objects, which contain a whole 'state' of a graphics API - as it stands, this translates to having one Vulkan instance per renderer, for example.
However, I am using volk to load Vulkan symbols. This library loads global functions based on the extensions enabled for a Vulkan instance, and problems arise when attempting to create multiple instances, potentially with different layers/extensions enabled.
For example, if I create an instance with debug utils enabled, the associated functions like vkCreateDebugUtilsMessengerEXT are loaded as expected. If I then create another instance without debug utils enabled, these functions seem to be unloaded, which sets their function pointers to NULL, causing a segfault when the instance that enabled them attempts to call them.
At the end of the day, supporting multiple Vulkan instances seems to be introducing a lot of issues - is there perhaps a better way of abstracting this design, or should I just limit the user to creating one Vulkan renderer (and therefore one instance) per program?


Answer:
Function pointers obtained through a particular VkInstance object (through vkGetInstanceProcAddr) cannot be used with different instance objects. This is just as true of the Vulkan SDK as it is of Volk. That is, a function pointer obtained as the result of calling volkLoadInstance(some_instance) can only be used on other some_instance objects or those derived from them (like VkDevices).
Note that Volk doesn't hand you function pointers; it initializes global function pointers. Therefore, if you call volkLoadInstance, all of the global functions it hands you are bound to that instance. You can call it again for a different instance, but you're just changing which global pointers you're talking about.
That is, Volk is specifically designed to be used with a single instance. It has functionality for easily using it with multiple devices without dynamic dispatch logic (volkLoadDeviceTable returns a table of function pointers specific to a device), but it has no similar construct for instances.
As such, trying to allow each "renderer" to have its own VkInstance object is going to run into problems. volkLoadInstance is not a fast function.
The reason Volk is written this way is because creating multiple instances like this is generally... not useful. Vulkan instance extensions tend to be about the interaction between your code and the operating system.
Consider VK_KHR_surface. This is an instance extension that acts as glue between your program and an OS window. If you want to render into a window, you need this instance extension. But the glue between the OS window (represented by a VkSurfaceKHR) and a Vulkan device is governed by VK_KHR_swapchain, which is a device extension. A device created from an instance doesn't have to use that extension. If it doesn't, then it won't be able to render into a surface.
But it doesn't harm the device in any way to have been created from an instance that uses VK_KHR_surface. So you should get your "renderers" together and set down a list of instance extensions that they could rely on, and then require them for the one VkInstance that they all share.

Broadly speaking, separate VkInstance objects exist primarily for DLLs, where there is a hard barrier between the global state of two pieces of code that happen to exist in the same program. One DLL can use a Vulkan instance, while the main program uses a separate one. The separate instances, and attendant global instance function pointers, make it easy for these two pieces of code to co-exist without stomping on each other.