Repository: conan-io/conan

Issue:
Symlinks in conan_package.tgz can have invalid nonzero size in the tar structure
Conan creates faulty tarfiles when there are symbolic links inside. This error is similar to https://bugs.python.org/issue1167128.

The symbolic links should have size 0 in the tarfile, but get a nonzero size, leading to partial package unpack during conan install.

In the tar contents below, the `243700` should have been all zeroes.

```
02ce4c00  6c 69 62 2f 62 66 64 2d  70 6c 75 67 69 6e 73 2f  |lib/bfd-plugins/|
02ce4c10  6c 69 62 6c 74 6f 5f 70  6c 75 67 69 6e 2e 73 6f  |liblto_plugin.so|
02ce4c20  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
02ce4c60  00 00 00 00 30 30 30 30  37 35 35 00 30 30 30 30  |....0000755.0000|
02ce4c70  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30  |000.0000000.0000|
02ce4c80  30 32 34 33 37 30 30 00  30 30 30 30 30 30 30 30  |0243700.00000000|
02ce4c90  30 30 30 00 30 33 30 31  34 32 00 20 32 2e 2e 2f  |000.030142. 2../|
02ce4ca0  2e 2e 2f 6c 69 62 65 78  65 63 2f 67 63 63 2f 78  |../libexec/gcc/x|
02ce4cb0  38 36 5f 36 34 2d 72 65  64 68 61 74 2d 6c 69 6e  |86_64-redhat-lin|
02ce4cc0  75 78 2d 67 6e 75 2f 35  2e 34 2e 30 2f 6c 69 62  |ux-gnu/5.4.0/lib|
02ce4cd0  6c 74 6f 5f 70 6c 75 67  69 6e 2e 73 6f 00 00 00  |lto_plugin.so...|
02ce4ce0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
02ce4d00  00 75 73 74 61 72 20 20  00 00 00 00 00 00 00 00  |.ustar  ........|
02ce4d10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
02ce4d40  00 00 00 00 00 00 00 00  00 30 30 30 30 30 30 30  |.........0000000|
02ce4d50  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
02ce4d60  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................
```

Untarring complains:

```
tar: Skipping to next header
tar: Error exit delayed from previous errors
```

The error is in the custom tar handling in `compress_files` in remote_manager.py: when treating symlinks, `info.size` should be set to 0. 

Using conan 1.9.2 on RHEL7.

To help us debug your issue please explain:

- [X] I've read the [CONTRIBUTING guide](https://raw.githubusercontent.com/conan-io/conan/develop/.github/CONTRIBUTING.md).
- [X] I've specified the Conan version, operating system version and any tool that can be relevant.
- [X] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.




Discussion:
Thanks for reporting, we will investigate it. The tar you have been debugging is the `conan_package.tgz` right?
Indeed, it is `conan_package.tgz`.
See PR https://github.com/conan-io/conan/pull/4051
Hello, I think this issue can be reopened. Today I've encountered the same problem with faulty package archives on version 1.14.0. After investigating, I've found that `compress_files` function now lives in *conans/client/cmd/uploader.py* but the code for it seems to be written before the provided fix. 

I've applied the [similar](https://github.com/conan-io/conan/pull/4051/commits/a5d9986abb76596b831c13b15fef5c35045155b3) fix and it everything worked.

Resolving patch:
diff --git a/conans/client/cmd/uploader.py b/conans/client/cmd/uploader.py
--- a/conans/client/cmd/uploader.py
+++ b/conans/client/cmd/uploader.py
@@ -502,6 +502,7 @@ def compress_files(files, symlinks, name, dest_dir, output=None):
             info = tarfile.TarInfo(name=filename)
             info.type = tarfile.SYMTYPE
             info.linkname = dest
+            info.size = 0  # A symlink shouldn't have size
             tgz.addfile(tarinfo=info)
 
         mask = ~(stat.S_IWOTH | stat.S_IWGRP)
@@ -516,6 +517,7 @@ def compress_files(files, symlinks, name, dest_dir, output=None):
             info.mode = os.stat(abs_path).st_mode & mask
             if os.path.islink(abs_path):
                 info.type = tarfile.SYMTYPE
+                info.size = 0  # A symlink shouldn't have size
                 info.linkname = os.readlink(abs_path)  # @UndefinedVariable
                 tgz.addfile(tarinfo=info)
             else:


Test patch:
diff --git a/conans/test/functional/configuration/compressed_symlinks_test.py b/conans/test/functional/configuration/compressed_symlinks_test.py
new file mode 100644
--- /dev/null
+++ b/conans/test/functional/configuration/compressed_symlinks_test.py
@@ -0,0 +1,53 @@
+import os
+import platform
+import unittest
+
+from conans.model.ref import ConanFileReference
+from conans.test.utils.tools import TestServer, TurboTestClient
+
+
+class CompressSymlinksZeroSize(unittest.TestCase):
+
+    @unittest.skipIf(platform.system() != "Linux", "Only linux")
+    def test_package_symlinks_zero_size(self):
+        server = TestServer()
+        client = TurboTestClient(servers={"default": server})
+
+        conanfile = """
+import os
+from conans import ConanFile, tools
+
+class HelloConan(ConanFile):
+
+    def package(self):
+        # Link to file.txt and then remove it
+        tools.save(os.path.join(self.package_folder, "file.txt"), "contents")
+        os.symlink("file.txt", os.path.join(self.package_folder, "link.txt"))    
+"""
+        ref = ConanFileReference.loads("lib/1.0@conan/stable")
+        # By default it is not allowed
+        pref = client.create(ref, conanfile=conanfile)
+        # Upload, it will create the tgz
+        client.upload_all(ref)
+
+        # We can uncompress it without warns
+        p_folder = client.cache.package_layout(pref.ref).package(pref)
+        tgz = os.path.join(p_folder, "conan_package.tgz")
+        client.run_command('gzip -d "{}"'.format(tgz))
+        client.run_command('tar tvf "{}"'.format(os.path.join(p_folder, "conan_package.tar")))
+        lines = str(client.out).splitlines()
+        """
+-rw-r--r-- 0/0               8 1970-01-01 01:00 file.txt
+lrw-r--r-- 0/0               0 1970-01-01 01:00 link.txt -> file.txt
+        """
+
+        self.assertIn("link.txt", " ".join(lines))
+        for line in lines:
+            if ".txt" not in line:
+                continue
+
+            size = int([i for i in line.split(" ") if i][2])
+            if "link.txt" in line:
+                self.assertEqual(int(size), 0)
+            elif "file.txt":
+                self.assertGreater(int(size), 0)