ocamllabs/ocaml-ctypes — pull request #175 by yallop (2014-08-20 15:42:59+00:00)
Title: Pointer paranoia

Use a safer representation of fat pointers internally, following suggestions by @pqwy.

Background: ctypes [represents a pointer as a record of three fields][static-ptr]:
* `reftyp`, the type of the pointed-to object
* `raw_ptr`, the raw address
* `pmanaged`, an OCaml value whose finaliser releases the pointed-to memory

In the past there was an additional field:
* `pbyte_offset`, an offset from `raw_ptr`, which was previously word-aligned

Passing a pointer value to C involves projecting out the `raw_ptr` field as, for instance, in the [`Cstubs_internals.raw_ptr`][current-raw_ptr] function, which is invoked by generated stub code.

A [(short-lived) version of `Cstubs_internals.raw_ptr`][raw_ptr-bug] had a bug. See if you can spot the problem:

```ocaml
let raw_ptr (CPointer { raw_ptr; pbyte_offset }) =
Ctypes_raw.PtrType.(add raw_ptr (of_int pbyte_offset))
```

In case you didn't spot it, the problem is as follows:
* `of_int` or `add` allocates a new object
* the allocation triggers a garbage collection
* the garbage collector observes that the argument to the `raw_ptr` function is no longer needed and collects it
* collecting the object runs the finaliser attached to `pmanaged`, freeing the memory pointed to by `raw_ptr`
* by the time `raw_ptr` is passed to C, the object it points to has been freed.

The bug didn't make it into a release, so the harm is limited (apart from wasting far too much of @pqwy's time). This patch aims to prevent similar bugs, by making it harder to inadvertently dissociate the `raw_ptr` value from the `pmanaged` field that manages the lifetime of the object. There are two steps:

* First, make [Static.cptr][static-ptr] into an abstract type, [Ctypes_ptr.Fat.t][fatptr] ([[1]](https://github.com/yallop/ocaml-ctypes/commit/bd15a4f), [[2]](https://github.com/yallop/ocaml-ctypes/commit/9453289), [[3]](https://github.com/yallop/ocaml-ctypes/commit/d0c5c40), [[4]](https://github.com/yallop/ocaml-ctypes/commit/e90f573)). Give the field that retrieves the raw address a [name][unsafe_raw_addr] that discourages careless use. Add a [macro][ctypes_addr_of_fatptr] for retrieving the `raw_ptr` field from C, where the explicit registration of roots with macros such as [`CAMLParam1`][camlparam] make it easier to control when objects can be collected.

* Next, remove almost all calls to `unsafe_raw_addr` ([[1]](https://github.com/yallop/ocaml-ctypes/commit/d5e2ef9), [[2]](https://github.com/yallop/ocaml-ctypes/commit/0a08c16), [[3]](https://github.com/yallop/ocaml-ctypes/commit/cd3ef96), [[4]](https://github.com/yallop/ocaml-ctypes/commit/6b93b75), [[5]](https://github.com/yallop/ocaml-ctypes/commit/2079dc4), [[6]](https://github.com/yallop/ocaml-ctypes/commit/33f01a6), [[7]](https://github.com/yallop/ocaml-ctypes/commit/5e1488c), [[8]](https://github.com/yallop/ocaml-ctypes/commit/6a5a043), [[9]](https://github.com/yallop/ocaml-ctypes/commit/99377e3), [[10]](https://github.com/yallop/ocaml-ctypes/commit/3c0882b), [[11]](https://github.com/yallop/ocaml-ctypes/commit/d72b42a), [[12]](https://github.com/yallop/ocaml-ctypes/commit/8d5e041), [[13]](https://github.com/yallop/ocaml-ctypes/commit/4536c22), [[14]](https://github.com/yallop/ocaml-ctypes/commit/0340fc9)). [Update the API documentation][unsafe-comment] for the [one call][remaining-call] to `unsafe_raw_addr` that remains to warn of the dangers.

Lots of commits here. Perhaps a rebase is due before merging.

[raw_ptr-bug]: https://github.com/ocamllabs/ocaml-ctypes/blob/23b3df61e2/src/cstubs/cstubs_internals.ml#L27-L28
[static-ptr]: https://github.com/ocamllabs/ocaml-ctypes/blob/76f0f4c57/src/ctypes/static.mli#L41-L43
[fatptr]: https://github.com/yallop/ocaml-ctypes/blob/8d5e041db3/src/ctypes/ctypes_ptr.ml#L27-L57
[ctypes_addr_of_fatptr]: https://github.com/yallop/ocaml-ctypes/blob/8d5e041db3/src/ctypes/raw_pointer.h#L28
[unsafe_raw_addr]: https://github.com/yallop/ocaml-ctypes/blob/8d5e041db3/src/ctypes/ctypes_ptr.ml#L47-L50
[camlparam]: http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual033.html#htoc274
[current-raw_ptr]: https://github.com/ocamllabs/ocaml-ctypes/blob/76f0f4c57/src/cstubs/cstubs_internals.ml#L26
[remaining-call]: https://github.com/yallop/ocaml-ctypes/blob/8d5e041db3/src/ctypes/memory.ml#L154
[unsafe-comment]: https://github.com/yallop/ocaml-ctypes/commit/1a4ed5ccf