cython/cython — pull request #191 by nnemkin (2013-03-03 07:43:50+00:00)
Title: Py_UNICODE* string support

Hi,

Here comes my proposal to add minimal support for zero-terminated `Py_UNICODE*` strings (and arrays) in the same way as `char*` strings are supported now.

The goal is to make interoperating with 2-byte wchat_t* Windows APIs as easy and transparent as it currently is with char* APIs. See my motivational example here: https://gist.github.com/nnemkin/5072774.
On other platforms, `Py_UNICODE*` strings are of little use, especially in the light of PEP393, but on Windows there are no alternatives.

Minimal support, just enough to avoid myriads of explicit `PyUnicode_XXX` calls, includes the following:
* Automatic conversion between `Py_UNICODE*` and `unicode`.
(with `PyUnicode_FromUnicode`/`PyUnicode_AsUnicode`).
* Slicing of `Py_UNICODE*` (same as above)
* Builtin function mapping (`len` -> `Py_UNICODE_strlen`).
* C-level `Py_UNICODE[]` literals with `u''` syntax.

... and *does not* include convenience features unrelated to interop:
* `in`, `not in` operators
* Iteration
* Optimized `.encode() -> bytes` method
* `std::wstring` support

The patch is supposed to be low-impact and fully backward compatible.
Slight changes to some often seen error messages are the worst I can think of.

Warning: this is my first attempt to modify Cython internals. For the most part I have just added Py_UNICODE* handling to some existing char* code paths.
Also, there are a few places that I'm not sure if they need to be modified: `AttributeNode`, `BinopNode`, `IntBinopNode`, `AddNode` in particular.