TUD-OS/NRE — pull request #25 by blitz (2012-11-14 18:10:45+00:00)
Title: Virtual IRQs (or Retiring Kernel Semaphores)

**Disclaimer: This is only meant as discussion and is not ready to be merged.**

I have implemented virtual IRQs for NOVA. The implementation is influenced by Fiasco.OC's virtual IRQs, but differs in some important ways. The goal is to have a simple event notification system in the kernel that supersedes kernel semaphores and simplifies vancouver.

Comments are welcome.

### New kernel mechanisms

Let me outline what the implementation currently provides: On creation time vIRQs are bound to one EC (the handler), which can wait (block) for events on all virtual IRQs that are connected to it.

vIRQ objects can be freely delegated and the only operation on them is to trigger them. Events are distinguished by sticking an event bitmask to each vIRQ object (also on creation time). If the vIRQ is triggered, this bitmask is OR-ed into the handling EC's UTCB. I repurposed utcb.tls (now named utcb.event), because no one uses it.

ECs can voluntarily block to wait for events. If events are pending, ECs will wakeup (or not block at all). The handling EC is then responsible to zero the relevant bits in its event bit mask in its UTCB. The block operation takes an additional mask parameter to limit wakeups to specific events.

There is also experimental support for adding another EC to the virtual IRQ that gets recalled whenever the handler EC is not actively blocking. Those two ECs may be the same or not. This feature is useful to implement asynchronous timeouts or events in Vancouver that need to be handled in VCPU context. This removes one kernel entry and context switch in Vancouver's usual IRQ injection path.

The code is in my NOVA repository in the virq branch. The syscall interface is not very beautiful at the moment and there is minor code duplication with the semaphore implementation, but the current goal is to make tracking upstream NOVA feasible for me.

### New userspace mechanisms

I implemented a userspace mutex built on top of virtual IRQs. It survives stress testing and may be used in all places where NRE currently uses userspace semaphores. I will try out the latter in the coming days and see what happens.

The implementation is in [include/kobj/Mutex.h](https://github.com/blitz/NRE/blob/virq/nre/include/kobj/Mutex.h).

### Pros/Cons

Pro:
- Threads can wait on an arbitrary number of events. Events are merged and threads will see them all at once, but still can distinguish what happened (if they want to)
- Mutexes are implemented in userland:
- cheap deadlock detection!
- easy to debug who waits for whom
- ...

Contra:
- you can get half the way with the current semaphore implementation
- you lose the ability for using kernel semaphores for cross-PD critical sections
- but no one uses it anyway!
- ...