Don't specify a gemset when entering the directory; IMO this is an environmental choice and should not be part of the repository.