Interact properly with otherwise directive

Also works for blank directives.
Add tests for to/from handling