Repository: conan-io/conan
Pull request: self.export_source_folder (based on layout.py)
Changelog: Fix: Renamed `self.base_source_folder` to `self.export_source_folder`. That variable was introduced to reference the folder where the `export_sources` are. Currently, they are copied to the `source` folder but might be changed in the future to avoid copying them, so `self.export_source_folder` will always point to the folder containing the `exports_sources`.
Docs: https://github.com/conan-io/docs/pull/2514

Similar than https://github.com/conan-io/conan/pull/11053
Close https://github.com/conan-io/conan/issues/10643

Warn: The `update_conandata` check is not possible anymore with this implementation.


Problem statement:
[feature] [2.0] exports_sources management
Currently, by default, in the cache, we are copying the `source folder` to the `build folder` unless the `no_copy_source=True`, but _always_, the `self.source_folder` and `self.build_folder` points to the right location.

It might be worth it to explore if the `exports_sources` could have a similar mechanism. Maybe, by default they could be copied to the `base source folder` but unless `self.no_copy_exports_sources=True`.  And of course, a `self.exports_sources_folder` will point to the right location. 

**Note**: Maybe we should start by declaring the `self.exports_sources_folder` variable pointing to the `folders.base_source` to have a "pointer" for the user that, for example, wants to know where are the patches exported and then implement the "optional" copy mechanism, because at the end of the day, typically if you export all the sources, you don't want to skip the copy and if you are just exporting some source files, the copy is cheap so the benefit might not worth it.

Hints from issue discussion:
Conan 1.46 introduced ``def base_source_folder(self):`` that points to the base source folder for this purpose. For two reasons:

- The ``exports_sources_folder`` was already used, with different meaning (used in ``export_sources()`` method to point to the cache), and it was creating conflicts
- The ``exports_source_folder`` was a bit confusing in the local flow.

But I agree the above seems reasonable, so lets think about it.
Probably this could be addressed later if necessary, I would move this to 2.X. This is enough for now.
Let's try to use `self.exports_sources_folder`, this variable already exists to point to the cache during and export.

Commit message: remove useless check at update_conandata

Patch to review:
diff --git a/conan/tools/files/conandata.py b/conan/tools/files/conandata.py
index 43bd7a12d44..1b7066ac509 100644
--- a/conan/tools/files/conandata.py
+++ b/conan/tools/files/conandata.py
@@ -11,8 +11,6 @@ def update_conandata(conanfile, data):
     this only works for updating the conandata on the export() method, it seems it would
     be plain wrong to try to change it anywhere else
     """
-    if not hasattr(conanfile, "export_folder") or conanfile.export_folder is None:
-        raise ConanException("The 'update_conandata()' can only be used in the 'export()' method")
     path = os.path.join(conanfile.export_folder, "conandata.yml")
     if os.path.exists(path):
         conandata = load(path)
diff --git a/conan/tools/files/patches.py b/conan/tools/files/patches.py
index 616b13c0544..dd3084be959 100644
--- a/conan/tools/files/patches.py
+++ b/conan/tools/files/patches.py
@@ -52,7 +52,7 @@ def patch(conanfile, base_path=None, patch_file=None, patch_string=None, strip=0
     if patch_file:
         # trick *1: patch_file path could be absolute (e.g. conanfile.build_folder), in that case
         # the join does nothing and works.
-        patch_path = os.path.join(conanfile.base_source_folder, patch_file)
+        patch_path = os.path.join(conanfile.export_sources_folder, patch_file)
         patchset = patch_ng.fromfile(patch_path)
     else:
         patchset = patch_ng.fromstring(patch_string.encode())
diff --git a/conans/client/cmd/export.py b/conans/client/cmd/export.py
index 03c6cd7f6e8..6af66ebc9ec 100644
--- a/conans/client/cmd/export.py
+++ b/conans/client/cmd/export.py
@@ -468,7 +468,9 @@ def export_source(conanfile, origin_folder, destination_source_folder):
     package_output = ScopedOutput("%s exports_sources" % output.scope, output)
     copier.report(package_output)
 
+    conanfile.folders.set_base_export_sources(destination_source_folder)
     _run_method(conanfile, "export_sources", origin_folder, destination_source_folder, output)
+    conanfile.folders.set_base_export_sources(None)
 
 
 def export_recipe(conanfile, origin_folder, destination_folder):
@@ -499,7 +501,9 @@ def export_recipe(conanfile, origin_folder, destination_folder):
         copier(pattern, links=True, excludes=excluded_exports)
     copier.report(package_output)
 
+    conanfile.folders.set_base_export(destination_folder)
     _run_method(conanfile, "export", origin_folder, destination_folder, output)
+    conanfile.folders.set_base_export(None)
 
 
 def _run_method(conanfile, method, origin_folder, destination_folder, output):
@@ -510,8 +514,6 @@ def _run_method(conanfile, method, origin_folder, destination_folder, output):
         output.highlight("Calling %s()" % method)
         copier = FileCopier([origin_folder], destination_folder)
         conanfile.copy = copier
-        folder_name = "%s_folder" % method
-        setattr(conanfile, folder_name, destination_folder)
         default_options = conanfile.default_options
         try:
             # TODO: Poor man attribute control access. Convert to nice decorator
@@ -521,6 +523,5 @@ def _run_method(conanfile, method, origin_folder, destination_folder, output):
                     export_method()
         finally:
             conanfile.default_options = default_options
-            delattr(conanfile, folder_name)
         export_method_output = ScopedOutput("%s %s() method" % (output.scope, method), output)
         copier.report(export_method_output)
diff --git a/conans/client/conan_api.py b/conans/client/conan_api.py
index 0452102242f..e59ccf05e67 100644
--- a/conans/client/conan_api.py
+++ b/conans/client/conan_api.py
@@ -831,6 +831,8 @@ def source(self, path, source_folder=None, info_folder=None, cwd=None):
         # only infos if exist
         conanfile = self.app.graph_manager.load_consumer_conanfile(conanfile_path, info_folder)
         conanfile.folders.set_base_source(source_folder)
+        conanfile.folders.set_base_export(source_folder)
+        conanfile.folders.set_base_export_sources(source_folder)
         conanfile.folders.set_base_build(None)
         conanfile.folders.set_base_package(None)
 
diff --git a/conans/client/installer.py b/conans/client/installer.py
index e6c14577e84..9f8ab1224a2 100644
--- a/conans/client/installer.py
+++ b/conans/client/installer.py
@@ -104,6 +104,8 @@ def _prepare_sources(self, conanfile, pref, package_layout, remotes):
         retrieve_exports_sources(self._remote_manager, self._cache, conanfile, pref.ref, remotes)
 
         conanfile.folders.set_base_source(source_folder)
+        conanfile.folders.set_base_export_sources(source_folder)
+        conanfile.folders.set_base_export(source_folder)
         conanfile.folders.set_base_build(None)
         conanfile.folders.set_base_package(None)
 
diff --git a/conans/model/conan_file.py b/conans/model/conan_file.py
index b33036f8f87..5e251c02b5f 100644
--- a/conans/model/conan_file.py
+++ b/conans/model/conan_file.py
@@ -258,13 +258,15 @@ def source_folder(self):
         return self.folders.source_folder
 
     @property
-    def base_source_folder(self):
-        """ returns the base_source folder, that is the containing source folder in the cache
-        irrespective of the layout() and where the final self.source_folder (computed with the
-        layout()) points.
-        This can be necessary in the source() or build() methods to locate where exported sources
-        are, like patches or entire files that will be used to complete downloaded sources"""
-        return self.folders._base_source
+    def export_sources_folder(self):
+        """points to the base source folder when calling source() and to the cache export sources
+        folder while calling the exports_sources() method. Prepared in case we want to introduce a
+        'no_copy_export_sources' and point to the right location always."""
+        return self.folders._base_export_sources
+
+    @property
+    def export_folder(self):
+        return self.folders._base_export
 
     @property
     def build_folder(self):
diff --git a/conans/model/layout.py b/conans/model/layout.py
index d8f5451412f..565186457e8 100644
--- a/conans/model/layout.py
+++ b/conans/model/layout.py
@@ -20,6 +20,8 @@ def __init__(self):
         self._base_package = None
         self._base_generators = None
         self._base_imports = None
+        self._base_export = None
+        self._base_export_sources = None
 
         self.source = ""
         self.build = ""
@@ -55,6 +57,9 @@ def set_base_folders(self, conanfile_folder, output_folder):
         self._base_generators = output_folder or base_folder
         self._base_imports = output_folder or base_folder
 
+        self._base_export = output_folder or base_folder
+        self._base_export_sources = output_folder or base_folder
+
     @property
     def source_folder(self):
         if self._base_source is None:
@@ -131,3 +136,9 @@ def base_imports(self):
 
     def set_base_imports(self, folder):
         self._base_imports = folder
+
+    def set_base_export(self, folder):
+        self._base_export = folder
+
+    def set_base_export_sources(self, folder):
+        self._base_export_sources = folder
diff --git a/conans/test/functional/layout/test_exports_sources.py b/conans/test/functional/layout/test_exports_sources.py
index 6c08b2c6e11..2602c70ebf7 100644
--- a/conans/test/functional/layout/test_exports_sources.py
+++ b/conans/test/functional/layout/test_exports_sources.py
@@ -25,7 +25,7 @@ def layout(self):
 
             def source(self):
                 save(self, "CMakeLists.txt", "old, bad") # EMULATE A DOWNLOAD!
-                base_source = self.base_source_folder
+                base_source = self.export_sources_folder
                 mypatch = load(self, os.path.join(base_source, "patches/mypatch"))
                 self.output.info("MYPATCH-SOURCE {}".format(mypatch))
                 shutil.copy(os.path.join(base_source, "CMakeLists.txt"),
@@ -35,7 +35,7 @@ def build(self):
                 path = os.path.join(self.source_folder, "CMakeLists.txt")
                 cmake = load(self, path)
                 self.output.info("MYCMAKE-BUILD: {}".format(cmake))
-                path = os.path.join(self.base_source_folder, "patches/mypatch")
+                path = os.path.join(self.export_sources_folder, "patches/mypatch")
                 cmake = load(self, path)
                 self.output.info("MYPATCH-BUILD: {}".format(cmake))
             """)
diff --git a/conans/test/integration/conanfile/conan_data_test.py b/conans/test/integration/conanfile/conan_data_test.py
index ff154c15cb3..dff646f87ed 100644
--- a/conans/test/integration/conanfile/conan_data_test.py
+++ b/conans/test/integration/conanfile/conan_data_test.py
@@ -298,22 +298,6 @@ def source(self):
         assert "pkg/0.1: 0.1-url: myurl!!" in c.out
         assert "pkg/0.1: 0.2-url: new!!" in c.out
 
-    def test_conandata_update_error(self):
-        """ test the update_conandata() helper fails if used outside export()
-        """
-        c = TestClient()
-        conanfile = textwrap.dedent("""
-            from conan import ConanFile
-            from conan.tools.files import update_conandata
-            class Pkg(ConanFile):
-                name = "pkg"
-                version = "0.1"
-                def source(self):
-                    update_conandata(self, {})
-            """)
-        c.save({"conanfile.py": conanfile})
-        c.run("create .", assert_error=True)
-        assert "The 'update_conandata()' can only be used in the 'export()' method" in c.out
 
     def test_conandata_create_if_not_exist(self):
         """ test the update_conandata() creates the file if it doesn't exist
diff --git a/conans/test/integration/layout/export_folder_variable_test.py b/conans/test/integration/layout/export_folder_variable_test.py
new file mode 100644
index 00000000000..f7e7b55a47d
--- /dev/null
+++ b/conans/test/integration/layout/export_folder_variable_test.py
@@ -0,0 +1,180 @@
+import textwrap
+
+from conans.model.ref import ConanFileReference
+from conans.test.utils.tools import TestClient
+
+
+class TestExportFoldersAvailability:
+
+    def test_export_sources_folder_availability_local_methods(self):
+        conanfile = textwrap.dedent('''
+        from conan import ConanFile
+
+        class ConanLib(ConanFile):
+
+            def layout(self):
+                self.folders.source = "MY_SOURCE"
+
+            def generate(self):
+                self.output.info("Running generate, value {}!".format(self.export_sources_folder))
+
+            def export(self):
+                self.output.info("Running export, value {}!".format(self.export_sources_folder))
+
+            def export_sources(self):
+                self.output.info("Running export_sources, value {}!".format(self.export_sources_folder))
+
+            def source(self):
+                self.output.info("Running source, value {}!".format(self.export_sources_folder))
+
+            def build(self):
+                self.output.info("Running build, value {}!".format(self.export_sources_folder))
+
+            def package(self):
+                self.output.info("Running package, value {}!".format(self.export_sources_folder))
+
+        ''')
+        client = TestClient()
+        client.save({"conanfile.py": conanfile})
+
+        client.run("export . foo/1.0@")
+        cache_exports_sources = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).export_sources()
+        assert "Running export, value None!" in client.out
+        assert "Running export_sources, value {}!".format(cache_exports_sources) in client.out
+
+        client.run("install .")
+        # This might be a bit unexpected but self.source_folder is also defined in generate()
+        # so it looks consistent with having declared the export_sources_folder
+        assert "Running generate, value {}!".format(client.current_folder) in client.out
+
+        client.run("source .")
+        assert "Running source, value {}!".format(client.current_folder) in client.out
+
+        client.run("build .")
+        assert "Running build, value {}!".format(client.current_folder) in client.out
+
+    def test_export_folder_availability_local_methods(self):
+        conanfile = textwrap.dedent('''
+        from conan import ConanFile
+
+        class ConanLib(ConanFile):
+
+            def layout(self):
+                self.folders.source = "MY_SOURCE"
+
+            def generate(self):
+                self.output.info("Running generate, value {}!".format(self.export_folder))
+
+            def export(self):
+                self.output.info("Running export, value {}!".format(self.export_folder))
+
+            def export_sources(self):
+                self.output.info("Running export_sources, value {}!".format(self.export_folder))
+
+            def source(self):
+                self.output.info("Running source, value {}!".format(self.export_folder))
+
+            def build(self):
+                self.output.info("Running build, value {}!".format(self.export_folder))
+
+            def package(self):
+                self.output.info("Running package, value {}!".format(self.export_folder))
+
+        ''')
+        client = TestClient()
+        client.save({"conanfile.py": conanfile})
+
+        client.run("export . foo/1.0@")
+        cache_exports = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).export()
+        assert "Running export_sources, value None!" in client.out
+        assert "Running export, value {}!".format(cache_exports) in client.out
+
+        client.run("install .")
+        # This might be a bit unexpected but self.source_folder is also defined in generate()
+        # so it looks consistent with having declared the export_sources_folder
+        assert "Running generate, value {}!".format(client.current_folder) in client.out
+
+        client.run("source .")
+        assert "Running source, value {}!".format(client.current_folder) in client.out
+
+        client.run("build .")
+        assert "Running build, value {}!".format(client.current_folder) in client.out
+
+    def test_export_folder_availability_create(self):
+        conanfile = textwrap.dedent('''
+        from conan import ConanFile
+
+        class ConanLib(ConanFile):
+
+            def layout(self):
+                self.folders.source = "MY_SOURCE"
+
+            def generate(self):
+                self.output.info("Running generate, value {}!".format(self.export_folder))
+
+            def export(self):
+                self.output.info("Running export, value {}!".format(self.export_folder))
+
+            def export_sources(self):
+                self.output.info("Running export_sources, value {}!".format(self.export_folder))
+
+            def source(self):
+                self.output.info("Running source, value {}!".format(self.export_folder))
+
+            def build(self):
+                self.output.info("Running build, value {}!".format(self.export_folder))
+
+            def package(self):
+                self.output.info("Running package, value {}!".format(self.export_folder))
+
+        ''')
+        client = TestClient()
+        client.save({"conanfile.py": conanfile})
+        cache_exports = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).export()
+        client.run("create . foo/1.0@")
+        assert "Running export_sources, value None!" in client.out
+        assert "Running export, value {}!".format(cache_exports) in client.out
+        cache_base_source = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).source()
+        assert "Running generate, value {}!".format(cache_base_source) in client.out
+        assert "Running source, value {}!".format(cache_base_source) in client.out
+        assert "Running build, value {}!".format(cache_base_source) in client.out
+
+
+    def test_export_sources_folder_availability_create(self):
+        conanfile = textwrap.dedent('''
+        from conan import ConanFile
+
+        class ConanLib(ConanFile):
+
+            def layout(self):
+                self.folders.source = "MY_SOURCE"
+
+            def generate(self):
+                self.output.info("Running generate, value {}!".format(self.export_sources_folder))
+
+            def export(self):
+                self.output.info("Running export, value {}!".format(self.export_sources_folder))
+
+            def export_sources(self):
+                self.output.info("Running export_sources, value {}!".format(self.export_sources_folder))
+
+            def source(self):
+                self.output.info("Running source, value {}!".format(self.export_sources_folder))
+
+            def build(self):
+                self.output.info("Running build, value {}!".format(self.export_sources_folder))
+
+            def package(self):
+                self.output.info("Running package, value {}!".format(self.export_sources_folder))
+
+        ''')
+        client = TestClient()
+        client.save({"conanfile.py": conanfile})
+        cache_exports = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).export_sources()
+        client.run("create . foo/1.0@")
+        assert "Running export, value None!" in client.out
+        assert "Running export_sources, value {}!".format(cache_exports) in client.out
+        cache_base_source = client.cache.package_layout(ConanFileReference.loads("foo/1.0")).source()
+        assert "Running generate, value {}!".format(cache_base_source) in client.out
+        assert "Running source, value {}!".format(cache_base_source) in client.out
+        assert "Running build, value {}!".format(cache_base_source) in client.out
diff --git a/conans/test/unittests/tools/files/test_patches.py b/conans/test/unittests/tools/files/test_patches.py
index 81d0b3b7f25..7f2958aeac7 100644
--- a/conans/test/unittests/tools/files/test_patches.py
+++ b/conans/test/unittests/tools/files/test_patches.py
@@ -38,6 +38,7 @@ def mock_fromstring(string):
 def test_single_patch_file(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.folders.set_base_source("/my_source")
+    conanfile.folders.set_base_export_sources("/my_source")
     conanfile.display_name = 'mocked/ref'
     patch(conanfile, patch_file='patch-file')
     assert mock_patch_ng.filename.replace("\\", "/") == '/my_source/patch-file'
@@ -49,6 +50,7 @@ def test_single_patch_file(mock_patch_ng):
 def test_single_patch_file_from_forced_build(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.folders.set_base_source("/my_source")
+    conanfile.folders.set_base_export_sources("/my_source")
     conanfile.display_name = 'mocked/ref'
     patch(conanfile, patch_file='/my_build/patch-file')
     assert mock_patch_ng.filename == '/my_build/patch-file'
@@ -60,6 +62,7 @@ def test_single_patch_file_from_forced_build(mock_patch_ng):
 def test_base_path(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.folders.set_base_source("/my_source")
+    conanfile.folders.set_base_export_sources("/my_source")
     conanfile.folders.source = "src"  # This not applies to find the patch file but for applying it
     conanfile.display_name = 'mocked/ref'
     patch(conanfile, patch_file='patch-file', base_path="subfolder")
@@ -72,6 +75,7 @@ def test_base_path(mock_patch_ng):
 def test_apply_in_build_from_patch_in_source(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.folders.set_base_source("/my_source")
+    conanfile.folders.set_base_export_sources("/my_source")
     conanfile.display_name = 'mocked/ref'
     patch(conanfile, patch_file='patch-file', base_path="/my_build/subfolder")
     assert mock_patch_ng.filename.replace("\\", "/") == '/my_source/patch-file'
@@ -85,6 +89,7 @@ def test_apply_in_build_from_patch_in_source(mock_patch_ng):
 def test_single_patch_string(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.folders.set_base_source("my_folder")
+    conanfile.folders.set_base_export_sources("my_folder")
     conanfile.display_name = 'mocked/ref'
     patch(conanfile, patch_string='patch_string')
     assert mock_patch_ng.string == b'patch_string'
@@ -97,6 +102,7 @@ def test_single_patch_arguments(mock_patch_ng):
     conanfile = ConanFileMock()
     conanfile.display_name = 'mocked/ref'
     conanfile.folders.set_base_source("/path/to/sources")
+    conanfile.folders.set_base_export_sources("/path/to/sources")
     patch(conanfile, patch_file='patch-file', strip=23, fuzz=True)
     assert mock_patch_ng.filename.replace("\\", "/") == '/path/to/sources/patch-file'
     assert mock_patch_ng.apply_args == ("/path/to/sources", 23, True)
diff --git a/conans/test/utils/mocks.py b/conans/test/utils/mocks.py
index 250f520d5d1..45997592a69 100644
--- a/conans/test/utils/mocks.py
+++ b/conans/test/utils/mocks.py
@@ -173,6 +173,7 @@ def __init__(self, shared=None, options=None, options_values=None):
         self._conan_env_values = EnvValues()
         self.folders = Folders()
         self.folders.set_base_source(".")
+        self.folders.set_base_export_sources(".")
         self.folders.set_base_build(".")
         self.folders.set_base_install("myinstallfolder")
         self.folders.set_base_generators(".")


Code review comments:
diff_hunk: @@ -11,8 +11,6 @@ def update_conandata(conanfile, data):
     this only works for updating the conandata on the export() method, it seems it would
     be plain wrong to try to change it anywhere else
     """
-    if not hasattr(conanfile, "export_folder") or conanfile.export_folder is None:
path: conan/tools/files/conandata.py
text: @user1:
This, we can still keep the ``if conanfile.export_folder is None``, if we invalidate it just after the ``export()`` method, can't we?

@author:
I think it is not that easy, currently (in this PR), the `conanfile.export_folder` is available in many other places, because [this](https://github.com/conan-io/conan/pull/11055/files#diff-c2ca7643fc907db00b41a346805bc9926f9a77adeaefebb10f42d5c528544536R107-R108) and [this](https://github.com/conan-io/conan/pull/11055/files#diff-c15f4aacf65aa58b31a041bac63e4dd7337355d9ec44a8f32b2a5815d5eb5ea9R834-R835) because we want to read them pointing to the "base source", so we don't know if it is set because you are in the "export" method or not.

@user1:
I don't fully get it. ``conanfile.export_folder`` is defined just before calling ``conanfile.export()`` method, and removed after it. Not used anywhere else. The ``export_folder`` shouldnt be pointing to anywhere else in any other method, because it is only relevant to the recipe, and the recipe is no longer copied to the "source" folder in the cache.  What am I missing?

@author:
Ok, if we don't want that variable to exist in any other place I guess it can be removed and the check restored.