Fix a ref count leak

Make sure to fail on the unsupported direction