To save time, I gathered some UNIX alternatives and nice architectures into a pasteable link:

https://news.ycombinator.com/item?id=10957020

Quite a few features have moved into mainstream. Often not as consistent as before. There's actually plenty more in the making over past decade. The comments here would make you think there's not innovation in CompSci but there is. JX is a nice, recent example where they do a capability/isolation-like architecture, implement it via language-based security in VM, make it faster than many microkernels, put half the drivers in VM, and run that & other parts of drivers on microkernel for lower risk. Already have a web server to show usefulness. Clever stuff that might be good in appliances maybe with MINIX 3-like self-healing.

For old ones, Burroughs stays top on my list given they scientifically designed a HW/SW combo from a language report (ALGOL) to be ideal machine for it. You usually worked closer to algorithm level instead of low-level with good reliability/security. KeyKOS had fine-grained isolation of everything plus persistence where your data... whole running system... would likely survive a crash. VMS's clustering and central support for clustered apps was so good that you basically just configure the boxes, tell the apps to use OS functions, set policy, and you're good to go. 17 years uptime on highest one. Convergent's CTOS and Tannenbaum's Amoeba both made a collection of workstations act like one mainframe or minicomputer OS with all resources shared for max utilization. We have Grid Computing but clean integration.

The most interesting stuff for developers and hackers was in LISP machines, esp Genera. Features from LISP that should go into every programming language: interactive running commands for testing; incremental, per-function compilation; live updates of that to running app for instant results. Combined with safe typing, this would let you crank out code like lightening. The OS itself was implemented with this high-level, highly-debuggable, live-updating language. It came with source. So, you could trace a problem in your app from it to system call seeing actual source code of OS along with current, running state of how it was being [mis]used. You could then correct the system if you chose to keep it running. Live updates and modification of your own running system with same language from apps all the way to bottom of OS is just unheard of today. Needless to say, they rarely crashed outside of hardware failure. :)

Far as mobile, most of the work is going into isolation and virtualization mechanisms, some with TrustZone, etc. OKL4 was best result there. I don't follow mobile as much & search engines are often clogged. Some Google wrangling got me these innovations.

https://engineering.purdue.edu/~xzl/xsel/papers/asplos12.pdf

Reflex addresses heterogenous programming of fast and low-power CPU's. Manual, tedious process. They applied old concept of software DSM to make it easy. I'm a DSM fan since supercomputer days so that's awesome.

http://askarov.net/oakland12.pdf

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.59....

https://arxiv.org/pdf/0707.0762.pdf

https://www.cs.cornell.edu/people/egs/papers/magnetos-mobisy...

Alright, I've thrown a few others above that illustrate different ways OS research is going. Last two are modern grid OS's with one quite innovative in that it works on ad hoc networks whose assumptions are as bad as WAN's. The 2nd one has lots of references to interesting systems. Should give you a taste of clever stuff going on in mobile and grid that you might have missed.