Fully interrupt driven solution was too slow. Rewrote to main loop state machine way.
Clarifications in schematic