Repository: jmcgeheeiv/pyfakefs
Pull request #490 (2019-06-28 15:18:09)

Added support for relative symlinks in add_real_directory
Currently ignores `add_real_directory` for directories and flattens them for files. This PR adds support for relative dir and file symlinks by calling `create_symlink` within `add_real_directory`.

luto: > OSError: [Errno 63] File name too long: '/private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-M7VUbK/build/lib/pyfakefs/tests/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/fixtures/symlink_dir_relative/all_tests_without_extra_packages.py'

ha :) I guess I'll have to create the symlinks dynamically during testing. That way I can also add support for absolute ones.

Before I do that, though, I'll wait for feedback :grin: 
luto: Thanks for the comments! I'll take a closer look on monday.
mrbean-bremen: Take your time! I was thinking about a new pypi release, as there is currently not much happening here, may be we will do that after this is finished (if you are not planning more changes, of course). 
luto: I gave a code a good overhaul, making much simpler. There is now support for absolute symlinks, symlinks pointing to outside the added directory and `lazy_read=True`. I also accidentally introduced a new public API `add_real_symlink`, which is used with lazy read mode. Please let me know what you think!
luto: The windows tests mostly work now (:tada:), except for [`test_resolve_nonexisting_file_after_36` on py27](https://ci.appveyor.com/project/jmcgeheeiv/pyfakefs/builds/25641630/job/b3u6cobmp7y7ol2q). I'm not quite sure what is happening there and would be very happy about any pointers :sweat_smile:
luto: > the minor typo in the doc string (missing "symlink")

fixed.

>the removal of the workaround in the unrelated failing test.

After looking at the pathlib2 issue as well as the git history of that test, I am not quite sure what the correct test would be. Additionally, I don't have a way to verify the behavior, since I'm lacking a windows system :/ Can you tackle that one, please?

> Thanks for the contribution!

Thanks for maintaining pyfakefs :grin: 
mrbean-bremen: > Can you tackle that one, please?

Yes, of course!

Patch:
diff --git a/CHANGES.md b/CHANGES.md
index b9bb756..049459d 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -14,6 +14,8 @@ The release versions are PyPi releases.
   * added argument `allow_root_user` to `Patcher` and `UnitTest` to allow
     forcing non-root access (see [#474](../../issues/474))
   * added basic support for `os.pipe` (see [#473](../../issues/473))
+  * added support for symlinks in `add_real_directory`
+  * added new public method `add_real_symlink`
   
 #### Infrastructure
   * added check for correctly installed Python 3 version in Travis.CI
diff --git a/pyfakefs/fake_filesystem.py b/pyfakefs/fake_filesystem.py
index b6479a6..2d64c2a 100644
--- a/pyfakefs/fake_filesystem.py
+++ b/pyfakefs/fake_filesystem.py
@@ -823,7 +823,9 @@ class FakeDirectoryFromRealDirectory(FakeDirectory):
             for entry in os.listdir(self.source_path):
                 source_path = os.path.join(self.source_path, entry)
                 target_path = os.path.join(base, entry)
-                if os.path.isdir(source_path):
+                if os.path.islink(source_path):
+                    self.filesystem.add_real_symlink(source_path, target_path)
+                elif os.path.isdir(source_path):
                     self.filesystem.add_real_directory(
                         source_path, self.read_only, target_path=target_path)
                 else:
@@ -2427,11 +2429,43 @@ class FakeFilesystem(object):
                                fake_file.st_dev)
         return fake_file
 
+    def add_real_symlink(self, source_path, target_path=None):
+        """Create a symlink at source_path (or target_path, if given).  It will
+        point to the same path as the symlink on the real filesystem.  Relative
+        symlinks will point relative to their new location.  Absolute symlinks
+        will point to the same, absolute path as on the real filesystem.
+
+        Args:
+            source_path: The path to the existing symlink.
+            target_path: If given, the name of the symlink in the fake
+                fileystem, otherwise, the same as `source_path`.
+
+        Returns:
+            the newly created FakeDirectory object.
+
+        Raises:
+            OSError: if the directory does not exist in the real file system.
+            OSError: if the symlink could not be created
+                (see :py:meth:`create_file`).
+            OSError: if on Windows before Python 3.2.
+            IOError: if the directory already exists in the fake file system.
+        """
+        source_path = self._path_without_trailing_separators(source_path)
+        if not os.path.exists(source_path) and not os.path.islink(source_path):
+            self.raise_os_error(errno.ENOENT, source_path)
+
+        target = os.readlink(source_path)
+
+        if target_path:
+            return self.create_symlink(target_path, target)
+        else:
+            return self.create_symlink(source_path, target)
+
     def add_real_directory(self, source_path, read_only=True, lazy_read=True,
                            target_path=None):
         """Create a fake directory corresponding to the real directory at the
         specified path.  Add entries in the fake directory corresponding to
-        the entries in the real directory.
+        the entries in the real directory.  Symlinks are supported.
 
         Args:
             source_path: The path to the existing directory.
@@ -2476,8 +2510,19 @@ class FakeFilesystem(object):
             for base, _, files in os.walk(source_path):
                 new_base = os.path.join(new_dir.path,
                                         os.path.relpath(base, source_path))
+                if self._is_link_supported():
+                    for fileEntry in os.listdir(base):
+                        abs_fileEntry = os.path.join(base, fileEntry)
+
+                        if not os.path.islink(abs_fileEntry):
+                            continue
+
+                        self.add_real_symlink(abs_fileEntry, os.path.join(new_base, fileEntry))
                 for fileEntry in files:
-                    self.add_real_file(os.path.join(base, fileEntry),
+                    path = os.path.join(base, fileEntry)
+                    if self._is_link_supported() and os.path.islink(path):
+                        continue
+                    self.add_real_file(path,
                                        read_only,
                                        os.path.join(new_base, fileEntry))
         return new_dir


Test patch:
diff --git a/pyfakefs/tests/fake_filesystem_test.py b/pyfakefs/tests/fake_filesystem_test.py
index b40b8bb..e8a1ad7 100644
--- a/pyfakefs/tests/fake_filesystem_test.py
+++ b/pyfakefs/tests/fake_filesystem_test.py
@@ -15,6 +15,7 @@
 
 """Unittest for fake_filesystem module."""
 
+import contextlib
 import errno
 import os
 import stat
@@ -1974,6 +1975,111 @@ class RealFileSystemAccessTest(TestCase):
             self.filesystem.exists(
                 os.path.join(self.root_path, 'pyfakefs', '__init__.py')))
 
+    @contextlib.contextmanager
+    def create_symlinks(self, symlinks):
+        for link in symlinks:
+            os.symlink(link[0], link[1])
+
+        yield
+
+        for link in symlinks:
+            os.unlink(link[1])
+
+    @unittest.skipIf(TestCase.is_windows and sys.version_info < (3, 3),
+                     'Links are not supported under Windows before Python 3.3')
+    def test_add_existing_real_directory_symlink(self):
+        fake_open = fake_filesystem.FakeFileOpen(self.filesystem)
+        real_directory = os.path.join(self.root_path, 'pyfakefs', 'tests')
+        symlinks = [
+            ('..', os.path.join(real_directory, 'fixtures', 'symlink_dir_relative')),
+            ('../all_tests.py', os.path.join(real_directory, 'fixtures', 'symlink_file_relative')),
+            (real_directory, os.path.join(real_directory, 'fixtures', 'symlink_dir_absolute')),
+            (os.path.join(real_directory, 'all_tests.py'), os.path.join(real_directory, 'fixtures', 'symlink_file_absolute')),
+            ('/etc/something', os.path.join(real_directory, 'fixtures', 'symlink_file_absolute_outside')),
+        ]
+
+        self.filesystem.create_file('/etc/something')
+
+        with fake_open('/etc/something', 'w') as f:
+            f.write('good morning')
+
+        with self.create_symlinks(symlinks):
+            self.filesystem.add_real_directory(real_directory, lazy_read=False)
+
+        for link in symlinks:
+            self.assertTrue(self.filesystem.islink(link[1]))
+
+        # relative
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_dir_relative')))
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_dir_relative/all_tests.py')))
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_file_relative')))
+
+        # absolute
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_dir_absolute')))
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_dir_absolute/all_tests.py')))
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_file_absolute')))
+
+        # outside
+        self.assertTrue(
+            self.filesystem.exists(
+                os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_file_absolute_outside')))
+        self.assertEqual(
+            fake_open(os.path.join(self.root_path, 'pyfakefs', 'tests',
+                             'fixtures/symlink_file_absolute_outside')).read(),
+            'good morning'
+        )
+
+    @unittest.skipIf(TestCase.is_windows and sys.version_info < (3, 3),
+                     'Links are not supported under Windows before Python 3.3')
+    def test_add_existing_real_directory_symlink_target_path(self):
+        real_directory = os.path.join(self.root_path, 'pyfakefs', 'tests')
+        symlinks = [
+            ('..', os.path.join(real_directory, 'fixtures', 'symlink_dir_relative')),
+            ('../all_tests.py', os.path.join(real_directory, 'fixtures', 'symlink_file_relative')),
+        ]
+
+        with self.create_symlinks(symlinks):
+            self.filesystem.add_real_directory(real_directory, target_path='/path', lazy_read=False)
+
+        self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_dir_relative'))
+        self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_dir_relative/all_tests.py'))
+        self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_file_relative'))
+
+    @unittest.skipIf(TestCase.is_windows and sys.version_info < (3, 3),
+                     'Links are not supported under Windows before Python 3.3')
+    def test_add_existing_real_directory_symlink_lazy_read(self):
+        real_directory = os.path.join(self.root_path, 'pyfakefs', 'tests')
+        symlinks = [
+            ('..', os.path.join(real_directory, 'fixtures', 'symlink_dir_relative')),
+            ('../all_tests.py', os.path.join(real_directory, 'fixtures', 'symlink_file_relative')),
+        ]
+
+        with self.create_symlinks(symlinks):
+            self.filesystem.add_real_directory(real_directory, target_path='/path', lazy_read=True)
+
+            self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_dir_relative'))
+            self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_dir_relative/all_tests.py'))
+            self.assertTrue(self.filesystem.exists('/path/fixtures/symlink_file_relative'))
+
     def test_add_existing_real_directory_tree_to_existing_path(self):
         self.filesystem.create_dir('/foo/bar')
         self.assert_raises_os_error(errno.EEXIST,