Set ownership for the correct directory