halide/Halide — pull request #569 by zvookin (2014-12-02 01:06:10+00:00)
Title: Gpu api naming

This pull request adds more structure to Halide's device runtime API. It is a breaking change for current users of the following API calls:
halide_dev_malloc
halide_dev_free
halide_copy_to_host
halide_copy_to_dev
halide_dev_sync
halide_release
and clients which replace CUDA, OpenCL, or OpenGL functionality via providing their own versions of the above or halide_dev_run, et al.

Previously, there was a single device per Halide filter, and in general per process, using Halide. This CL allows using multiple different device backends in a single filter. E.g. OpenCL or CUDA and OpenGL in one filter.

The names of the routines in each device's runtime are changed to be unique by incorporating the name of the backend into the routine name. This allows linking more than one backend into the same object file. The names are made regular by prefixing each with "halide__". E.g. "halide_cuda_init_kernels."

Routines which operate on buffer_t which already has GPU memory allocated dispatch to a device backend via a table of function pointers, called a "device interface" and represented by the struct halide_device_interface. In particular, the dev field of buffer_t is now a pointer to a struct:

struct device_handle_wrapper {
uint64_t device_handle;
const halide_device_interface *interface;
};

This carries both the pointer or device specific ID for the device memory held by the buffer and an interface identifying *which* device it is. This allows doing generic operations on the buffer, such as halide_device_free, even when there are multiple device backends in a single filter.

Runtime routines have been renamed to use "_device" uniformly instead of "_dev". (Syntactic changes were going to be required anyway due to the requirement to pass the device interface.)

The dirty bit handling has been moved into the device_interface layer and is no longer handled in device backend specific logic. The atomicity of various routines is guaranteed by the device_interface layer. In particular, halide_copy_to_host, and halide_copy_to_device and kernel execution started by Halide are guaranteed to be atomic with respect to each other, including the setting or clearing of the dirty bits via these mechanisms. The current method by which atomicity is guaranteed is too coarse and we should either come up with a method that uses one of a number of locks based on a hash of the bits of the buffer_t pointer, or another method. It may also be that halide_device_malloc and halide_device_free need to be incorporated into these atomicity guarantees.

(I started working on specifying a state machine for all valid buffer_t states and considering an API which allows e.g. allocating the host memory for a buffer_t and easily having a buffer_t with no host memory. Ideally such an interface would allow transitioning a buffer_t from any valid state to any other valid state atomically.)

Each device API now has its own runtime header file exported to Halide end users. This is necessary to get the properly named function prototypes for replacing the entire API via the weak function replacement mechanism. It also allows customizing certain function signatures, e.g. (halide__run), to be different per device API and to add extra functionality such as halide_opengl_output_client_bound and the OpenCL platform and GPU type setter/getter functions.

An open issue is what to do with halide_release. With these changes, one must pass the device interface to halide_release. (See the ugly #ifdefs in e.g. test/generator/gpu_object_lifetime_aottest.cpp .) It is useful to be able to release just one backend, but it would be nice to close down all backends at once. Alas, doing so requires some sort of dynamic registration of each backend that is actually used. It seems hard to do this without constructors. I consider this an open issue with the PR.

The runtime specific headers (HalideRuntimeCuda.h, HalideRuntimeOpenCL.h, and HalideRuntimeOpenGL.h) now include functions for wrapping and unwrapping user supplied GPU memory references in buffer_t structure and getting the current GPU memory reference from a buffer_t. These are useful for situations where application code does its own GPU work and needs to interoperate with Halide.

All GPU backed internal functions should now be in a namespace specific to the GPU backend. E.g. Halide::Runtime::Internal::Cuda .

I moved some pieces of the Makefile logic for the apps subdirectories into the commonly included Makefile. These should compile more reliably now.

The OS X OpenGL support now uses CGL functions rather than the very dated and deprecated AGL ones. (The code is currently #ifdef'ed, but that should probably be removed.)

Some tests for multiple GPU APIs being used in a single filter are needed before this is merged.