I've spoken about this before on here. We developed on HoloLens for a couple months. Working on the HoloLens app was actually my first foray into 3D development, and also required converting ThreeJS JSON into Unity models which was a mess.

The user experience
--------------

HoloLens is mesmerizing. I'm not big into VR or anything, and will often make the arguement that VR hype will die out and is a fad. But there's something very different about what Microsoft is doing. The ability to incorporate reality as a first class citizen in your 3D applications (or vice versa) is groundbreaking. People often complain about the FOV when they first try it out, and I had the same complaint, but your brain is able to compensate once it gets used to it, and then you stop noticing it. That's something you don't get from a short trial of it at a tech demo. The user inputs are indeed very clumsy still. We'll need vast improvements in this area before HoloLens can feel immersive. But the amazing thing is that this first pass isn't that bad. It can track your hands and it's a computer that sits on your head. I mean, come on! I'm only 22 and even I think that's amazing.

The developer experience
------------------------

One of the major short comings of HoloLens development is its dependency on Unity. C# isn't the problem. I love C# and use it daily now for web development. The problem is Unity uses .NET 2.0, and good luck finding C# libraries that are compatible. So for every new thing you want to do, you're going to have to find a "Unity compatible" C# library, which is very annoying.

Unity will work for what you need most of the time, but it turns out if you want to try something custom (like your own gestures) then you're out of luck, because the Unity APIs are limited in that way.

I suppose I'm mostly just not a fan of Unity's component model. Constantly switching between adjusting settings in the IDE and coding feels like a bad way of developing.

Okay, so maybe you want to try something a little lower level. Microsoft offers a C++ API as well, and for the most part this is what you want if you need to harness the limited power of the HoloLens. I haven't played around with all of the APIs, but I know of one in particular that left a bad taste in my mouth (this applies to Unity too) -- the spatial anchor API. For those of you who are unfamiliar, the spatial anchor API is the only way to acquire a durable and persistent reference to a real world location. This is done (I think) with sensor data (orientation, lighting, and images captured by the 4 on board spatial mapping cameras.) This is really an incredible feat of engineering, however it produces a binary which is around 15MB. Far too large to store in a database at scale. I'd like to see MS open up raw access to those sensors so middleware developers can try their hand at improving this aspect of HoloLens.

If C++ isn't your thing, there's a library called HoloJS. You guessed it, it's a JS runtime for HoloLens with access to native libs. I actually started my own variation on this (called HolographicJS) before Microsoft released theirs, but I'm happy they've taken over.

The future
----------

So what does this all mean for a device that seemly has its share of problems to overcome? Well, after trying it I'm fairly confident that MR as Microsoft calls it, is here to stay. The ability to mix reality with virtual reality, and augment that with a layer of environmental understanding is really incredible. I think we're just scratching the surface of the possiblities.

HoloLens is the first in a new field of devices that I believe will come to replace all forms of computers we currently use: phones, laptops, desktops, tablets, etc. Even things like IOT devices. Why spend time building your own interfaces when you can just augment the users'?

If v2 had better FOV and improved input tracking, I'd consider it a major success. But if it also included improved spatial mapping and a reliable GPS, that could bring us into a whole new world, quite literally.

The way I see it, the first company to solve outdoor use of an MR device, and solve what I'm calling the "universal spatial map" problem, will run the world of tomorrow.

Imagine every machine being capable of interfacing with you without the need for a screen or separate device. Imagine walking down the street, gesturing to a restaurant and placing an order before you even get inside.

Further down the line. What if we could transfer consciousness out of a dying car crash survivor into a computer. What if that person could then be virtually transferred back to the scene of the accident, to be greeted by those who are augmented.

Anyway, that's all crazy futurism; but the point is that reality starts with what is being done with HoloLens, and I think it's an incredible thing to be a part of.

To me, HoloLens feels like the Apple II.