I agree, each brand of device is vastly different, especially when it comes to input. There's a common core that needs to be handled at a low level under the hood, but in an extensible manner. Unity has already announced their intention of tackling this terribly difficult problem, but now we're just waiting for them to deliver on their promises.

They're the only ones in the position to really solve the problem at the right level, since it interacts so deeply with their rendering and input system, which is all under the hood so not possible for external developers modify, hook into, or fix problems.

Unity needs to integrate the input system with their new UGUI user interface toolkit, and integrate it with VR devices so all the different kinds of input devices plug in and are handled consistently. So they need to get very friendly and cooperative with all the different VR and input device hardware manufacturers they support, and do whatever it takes including flattering, shaming or partying them into cooperating if they foolishly decide they'd rather lock their developers into writing code specifically for their device, so that it doesn't work with other manufacturer's devices on purpose. (Here's looking at you, Apple!)

Unity has announced a new input system [1] and published a prototype [2], and asked for developers to give them feedback [3], but it's way too early to use in a product.

I'm optimistic that they mean what they say, and glad they're publishing the prototype and asking for feedback from developers, but I appreciate it's a difficult problem that will take a while to get right.

I've read over the documentation, and it seems to take a nice modular approach that can abstract away many differences between input devices, but it's just a hard problem by its very nature, and there will always be special circumstances for particular input devices that you need to handle on a case-by-case basis. So both the VR API and the input API should support hooks and customizations and plug-ins for special purpose hardware, and ways of querying capabilities (like position tracking), enumerating devices, reflecting on and hooking into the model.

I'm disappointed that neither the new UGUI user interface system nor the new input system prototype seem to fully support multitouch input and gestures in a nice way, like the TouchScript library does [4], but I hope they eventually support that as well.

One frustrating example of a bad API that both the Oculus and Unity SDKs have is the "recenter" call [5], which resets some hidden state inside the tracking code so the current direction of your head is "forward". You can't read and write the current yaw nulling offset [6], you can just "recenter", whatever that means. So there's no way to smoothly animate to recenter -- it always jerks your head around. They should expose that hidden state, and let me read and write it, so I can recenter smoothly. It's never a good idea to have a bunch of magic and hidden state behind an API like that. Plus the documentation is terrible -- they don't define what any of the terms mean, what the model is, or what the actual effect is mathematically. (Does it reset the yaw around the neck, resulting in a discontinuous jump in eye position? How can I tell what those measurements are, or know what the model really is? Does recenter work differently with devices that track the actual absolute position of your head, as opposed to estimating it from a model of the eye position relative to the neck?)

The input system should also support other kinds of gesture recognition like motion tracking (staring, tilting, pecking, shaking and nodding your head), popular "quantified self" input devices like the ShakeWeight with its optional heart rate monitor [7], network and virtual device adaptation, emulating and debugging hardware devices in the editor, etc. But right now it's so early in the game that the new input system prototype isn't yet integrated with the new VR API or the (less) new UGUI toolkit.

Oculus has examples of how to integrate head pointed ray casting with UGUI using reticles and cursors, but that code is brittle and dependent on their particular SDK and app-specific utilities, and it copies and modifies a bunch of the Unity UGUI input tracking code instead of subclassing and hooking into it, because Unity didn't expose the public virtual methods and hooks that they needed. They've beseeched Unity to fix that by making their API more public and hookier, but for now, Oculus's example UGUI integration code is pretty hacky, complex, and not a long term solution.

All the input tracking stuff will work much better when Unity fixes it under the hood (and lets developers open up the hood and hot-rod the fuck out of it, or shop for competing solutions on the asset store), instead of having 6 different VR and multitouch SDKs that all solve it in 6 different but overlapping ways, which you can't integrate together.

[1] http://blogs.unity3d.com/2016/04/12/developing-the-new-input...

[2] https://sites.google.com/a/unity3d.com/unity-input-advisory-...

[3] http://forum.unity3d.com/threads/welcome-new-input-system-re...

[4] http://touchscript.github.io/

[5] http://docs.unity3d.com/ScriptReference/VR.InputTracking.Rec...

[6] see "nulling problem" -- this is a GREAT paper: http://www.billbuxton.com/lexical.html

[7] https://www.youtube.com/watch?v=JImGs2Xysxs