Repository: reframe-hpc/reframe

Issue:
Allow Overriding the Default Mount Point of the Stage Directory in Containers
**Description**

Currently, ReFrame mounts the test’s stage directory inside the container to `/rfm_workdir` by default. This behavior is hard-coded and does not provide flexibility for users to specify an alternative mount point. - see https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.containers.ContainerPlatform.workdir

**Proposes feature**

Introduce a configuration option that allows users to override the stage directory as the default mount point that points to `/rfm_workdir` directory inside the container. 

**Motivation**

In environments with non-shared filesystems, input files are not located on the login node where the stage directory resides. As a result, the default mounting behavior restricts the accessibility of input files within the container. In this case one has to specify and additional mount point and and also set the `self.container_platform.workdir`. It would be more efficient to allow the possibility of changing the default mount point in such situations

Discussion:
@vkarak: I am not sure that this is actually needed, since you can still achieve the desired behaviour by setting two variables instead of one. Am I missing something?@mredenti: So you can definitely mount any other directory anywhere else and also change the default working directory inside the container. 
I guess it is just a matter of redundancy, why mount something in the container if you do not need it. 

Mounting two host directories to `/rfm_workdir` makes we wonder whether this is defined behaviour. I need to check whether container engines allow this or the second one specified takes precedence. In this case you would not be able to mount into  `/rfm_workdir` but it would be nice to keep a certain consistency for all tests such that the destination directory inside the container can always be 
 `/rfm_workdir` but need not be the stage directory being mounted@vkarak: You would still need the stagedir mounted inside the container if you wanted to copy out some files for inspection. I guess in your example, everything is in stdout so you don't need this. Maybe a variable defined as:

```python
mount_stagedir = variable(str, type(None), value='/rfm_workdir')
```

would make sense. If `None`, the stage directory would not be mounted at all. Afaiu this would work nicely for your case, wouldn't it?@mredenti: Yes, that would work nicely 

Solution patch:
diff --git a/reframe/core/containers.py b/reframe/core/containers.py
index dd7a9898c..0059ff79b 100644
--- a/reframe/core/containers.py
+++ b/reframe/core/containers.py
@@ -5,15 +5,20 @@
 
 import abc
 
-import reframe.core.fields as fields
 import reframe.utility as util
 import reframe.utility.typecheck as typ
+from reframe.core.meta import RegressionTestMeta
 
 
-_STAGEDIR_MOUNT = '/rfm_workdir'
+class _ContainerMeta(typ.ConvertibleType, RegressionTestMeta, abc.ABCMeta):
+    '''Metaclass for container platforms.
+
+    This essentially combines the ABCMeta with our standard metaclass that
+    provides the variables.
+    '''
 
 
-class ContainerPlatform(abc.ABC):
+class ContainerPlatform(abc.ABC, metaclass=_ContainerMeta):
     '''The abstract base class of any container platform.'''
 
     #: The default mount location of the test case stage directory inside the
@@ -23,7 +28,7 @@ class ContainerPlatform(abc.ABC):
     #:
     #: :type: :class:`str` or :class:`None`
     #: :default: :class:`None`
-    image = fields.TypedField(str, type(None))
+    image = variable(str, type(None), value=None)
 
     #: The command to be executed within the container.
     #:
@@ -36,7 +41,7 @@ class ContainerPlatform(abc.ABC):
     #:
     #: :type: :class:`str` or :class:`None`
     #: :default: :class:`None`
-    command = fields.TypedField(str, type(None))
+    command = variable(str, type(None), value=None)
 
     #: Pull the container image before running.
     #:
@@ -46,7 +51,7 @@ class ContainerPlatform(abc.ABC):
     #:
     #: :type: :class:`bool`
     #: :default: ``True``
-    pull_image = fields.TypedField(bool)
+    pull_image = variable(typ.Bool, value=True)
 
     #: List of mount point pairs for directories to mount inside the container.
     #:
@@ -57,34 +62,45 @@ class ContainerPlatform(abc.ABC):
     #:
     #: :type: :class:`list[tuple[str, str]]`
     #: :default: ``[]``
-    mount_points = fields.TypedField(typ.List[typ.Tuple[str, str]])
+    mount_points = variable(typ.List[typ.Tuple[str, str]], value=[])
+
+    #: Mount the stage directory inside the container
+    #:
+    #: If not :obj:`None`, the stage directory will always be mounted inside
+    #: the container in the specified location.
+    #:
+    #: .. versionadded:: 4.8
+    #:
+    #: :type: :class:`str` or :obj:`None`
+    #: :default: ``/rfm_workdir``
+    mount_stagedir = variable(str, type(None), value='/rfm_workdir')
 
     #: Additional options to be passed to the container runtime when executed.
     #:
     #: :type: :class:`list[str]`
     #: :default: ``[]``
-    options = fields.TypedField(typ.List[str])
+    options = variable(typ.List[str], value=[])
 
     #: The working directory of ReFrame inside the container.
     #:
-    #: This is the directory where the test's stage directory is mounted inside
-    #: the container. This directory is always mounted regardless if
-    #: :attr:`mount_points` is set or not.
+    #: By default, this is the directory where the test's stage directory is
+    #: mounted inside the container.
     #:
     #: :type: :class:`str`
     #: :default: ``/rfm_workdir``
     #:
     #: .. versionchanged:: 3.12.0
     #:    This attribute is no more deprecated.
-    workdir = fields.TypedField(str, type(None))
+    #:
+    #: .. versionchanged:: 4.8
+    #:    Whether the stage directory will be mounted or not is now controlled
+    #:    by the :attr:`mount_stagedir` variable. If it is not mounted, make
+    #:    sure to properly update also the working directory.
+    workdir = variable(str, type(None), value='/rfm_workdir')
 
-    def __init__(self):
-        self.image = None
-        self.command = None
-        self.workdir = _STAGEDIR_MOUNT
-        self.mount_points  = []
-        self.options = []
-        self.pull_image = True
+    @classmethod
+    def __rfm_cast_str__(cls, s):
+        return cls.create(s)
 
     @abc.abstractmethod
     def emit_prepare_commands(self, stagedir):
@@ -157,7 +173,10 @@ def emit_prepare_commands(self, stagedir):
 
     def launch_command(self, stagedir):
         super().launch_command(stagedir)
-        mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)]
+        mount_points = self.mount_points
+        if self.mount_stagedir:
+            mount_points.append((stagedir, self.mount_stagedir))
+
         run_opts = [f'-v "{mp[0]}":"{mp[1]}"' for mp in mount_points]
         if self.workdir:
             run_opts.append(f'-w {self.workdir}')
@@ -178,11 +197,10 @@ class Sarus(ContainerPlatform):
     #:
     #: :type: boolean
     #: :default: :class:`False`
-    with_mpi = fields.TypedField(bool)
+    with_mpi = variable(typ.Bool, value=False)
 
     def __init__(self):
         super().__init__()
-        self.with_mpi = False
         self._command = 'sarus'
 
     def emit_prepare_commands(self, stagedir):
@@ -197,7 +215,10 @@ def emit_prepare_commands(self, stagedir):
 
     def launch_command(self, stagedir):
         super().launch_command(stagedir)
-        mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)]
+        mount_points = self.mount_points
+        if self.mount_stagedir:
+            mount_points.append((stagedir, self.mount_stagedir))
+
         run_opts = [f'--mount=type=bind,source="{mp[0]}",destination="{mp[1]}"'
                     for mp in mount_points]
         if self.with_mpi:
@@ -238,11 +259,10 @@ class Singularity(ContainerPlatform):
     #:
     #: :type: boolean
     #: :default: :class:`False`
-    with_cuda = fields.TypedField(bool)
+    with_cuda = variable(typ.Bool, value=False)
 
     def __init__(self):
         super().__init__()
-        self.with_cuda = False
         self._launch_command = 'singularity'
 
     def emit_prepare_commands(self, stagedir):
@@ -250,7 +270,10 @@ def emit_prepare_commands(self, stagedir):
 
     def launch_command(self, stagedir):
         super().launch_command(stagedir)
-        mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)]
+        mount_points = self.mount_points
+        if self.mount_stagedir:
+            mount_points.append((stagedir, self.mount_stagedir))
+
         run_opts = [f'-B"{mp[0]}:{mp[1]}"' for mp in mount_points]
         if self.with_cuda:
             run_opts.append('--nv')
@@ -277,14 +300,3 @@ class Apptainer(Singularity):
     def __init__(self):
         super().__init__()
         self._launch_command = 'apptainer'
-
-
-class ContainerPlatformField(fields.TypedField):
-    def __init__(self, *other_types):
-        super().__init__(ContainerPlatform, *other_types)
-
-    def __set__(self, obj, value):
-        if isinstance(value, str):
-            value = ContainerPlatform.create(value)
-
-        return super().__set__(obj, value)
diff --git a/reframe/core/pipeline.py b/reframe/core/pipeline.py
index dfacb87ab..476033722 100644
--- a/reframe/core/pipeline.py
+++ b/reframe/core/pipeline.py
@@ -34,7 +34,7 @@
 import reframe.utility.udeps as udeps
 from reframe.core.backends import getlauncher, getscheduler
 from reframe.core.buildsystems import BuildSystemField
-from reframe.core.containers import (ContainerPlatform, ContainerPlatformField)
+from reframe.core.containers import ContainerPlatform
 from reframe.core.deferrable import (_DeferredExpression,
                                      _DeferredPerformanceExpression)
 from reframe.core.environments import Environment
@@ -466,8 +466,8 @@ def pipeline_hooks(cls):
     #: .. versionchanged:: 3.12.0
     #:    This field is now set automatically from the current partition's
     #:    configuration.
-    container_platform = variable(field=ContainerPlatformField,
-                                  value=_NoRuntime(), loggable=False)
+    container_platform = variable(ContainerPlatform, value=_NoRuntime(),
+                                  loggable=False, allow_implicit=True)
 
     #: .. versionadded:: 3.0
     #:
@@ -673,11 +673,11 @@ def pipeline_hooks(cls):
     #: the system/partition combinations.
     #: The reference itself is a four-tuple that contains the reference value,
     #: the lower and upper thresholds, and the measurement unit.
-    #: 
+    #:
     #: For non-zero reference values, lower and upper thresholds are
     #: percentages -/+ from the reference value in decimal form.
     #:
-    #: When a reference value of ``0`` is expected, lower and upper 
+    #: When a reference value of ``0`` is expected, lower and upper
     #: thresholds are interpreted as absolute values.
     #:
     #: An example follows:


Test patch:
diff --git a/unittests/test_containers.py b/unittests/test_containers.py
index 8dc7f5860..c2d313634 100644
--- a/unittests/test_containers.py
+++ b/unittests/test_containers.py
@@ -6,6 +6,7 @@
 import pytest
 
 import reframe.core.containers as containers
+from reframe.core.containers import ContainerPlatform
 
 
 @pytest.fixture(params=[
@@ -127,6 +128,74 @@ def expected_cmd_mount_points(container_variant):
                 '-B"/path/two:/two" -B"/foo:/rfm_workdir" '
                 '--pwd /rfm_workdir image:tag')
 
+@pytest.fixture
+def expected_cmd_mount_points_no_stagedir(container_variant):
+    if container_variant in {'Docker', 'Docker+nopull'}:
+        return ('docker run --rm -v "/path/one":"/one" '
+                '-v "/path/two":"/two" '
+                '-w /workspace image:tag cmd')
+    elif container_variant == 'Docker+nocommand':
+        return ('docker run --rm -v "/path/one":"/one" '
+                '-v "/path/two":"/two" '
+                '-w /workspace image:tag')
+    elif container_variant in {'Sarus', 'Sarus+nopull'}:
+        return ('sarus run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                '-w /workspace image:tag cmd')
+    elif container_variant == 'Sarus+nocommand':
+        return ('sarus run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                '-w /workspace image:tag')
+    elif container_variant == 'Sarus+mpi':
+        return ('sarus run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                '--mpi -w /workspace image:tag cmd')
+    elif container_variant == 'Sarus+load':
+        return ('sarus run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                '-w /workspace load/library/image:tag cmd')
+    elif container_variant in {'Shifter', 'Shifter+nopull'}:
+        return ('shifter run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                'image:tag cmd')
+    elif container_variant == 'Shifter+nocommand':
+        return ('shifter run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                'image:tag')
+    elif container_variant == 'Shifter+mpi':
+        return ('shifter run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                '--mpi image:tag cmd')
+    elif container_variant == 'Shifter+load':
+        return ('shifter run '
+                '--mount=type=bind,source="/path/one",destination="/one" '
+                '--mount=type=bind,source="/path/two",destination="/two" '
+                'load/library/image:tag cmd')
+    elif container_variant == 'Singularity':
+        return ('singularity exec -B"/path/one:/one" '
+                '-B"/path/two:/two" --pwd /workspace image:tag cmd')
+    elif container_variant == 'Singularity+cuda':
+        return ('singularity exec -B"/path/one:/one" '
+                '-B"/path/two:/two" --nv --pwd /workspace image:tag cmd')
+    elif container_variant == 'Singularity+nocommand':
+        return ('singularity run -B"/path/one:/one" '
+                '-B"/path/two:/two" --pwd /workspace image:tag')
+    elif container_variant == 'Apptainer':
+        return ('apptainer exec -B"/path/one:/one" '
+                '-B"/path/two:/two" --pwd /workspace image:tag cmd')
+    elif container_variant == 'Apptainer+cuda':
+        return ('apptainer exec -B"/path/one:/one" '
+                '-B"/path/two:/two" --nv --pwd /workspace image:tag cmd')
+    elif container_variant == 'Apptainer+nocommand':
+        return ('apptainer run -B"/path/one:/one" '
+                '-B"/path/two:/two" --pwd /workspace image:tag')
 
 
 @pytest.fixture
@@ -221,6 +290,16 @@ def test_mount_points(container_platform, expected_cmd_mount_points):
     assert cmd == expected_cmd_mount_points
 
 
+def test_mount_points_no_stagedir(container_platform,
+                                  expected_cmd_mount_points_no_stagedir):
+    container_platform.mount_points = [('/path/one', '/one'),
+                                       ('/path/two', '/two')]
+    container_platform.mount_stagedir = None
+    container_platform.workdir = '/workspace'
+    cmd = container_platform.launch_command('/foo')
+    assert cmd == expected_cmd_mount_points_no_stagedir
+
+
 def test_prepare_command(container_platform, expected_cmd_prepare):
     commands = container_platform.emit_prepare_commands('/foo')
     assert commands == expected_cmd_prepare
@@ -304,3 +383,14 @@ def test_run_with_workdir(container_platform_with_opts,
     container_platform_with_opts.workdir = 'foodir'
     found_commands = container_platform_with_opts.launch_command('/foo')
     assert found_commands == expected_run_with_workdir
+
+
+@pytest.fixture(params=['Docker', 'Sarus', 'Shifter',
+                        'Singularity', 'Apptainer'])
+def container_platform_name(request):
+    return request.param
+
+
+def test_create_from_str(container_platform_name):
+    cp = ContainerPlatform(container_platform_name)
+    assert type(cp).__name__ == container_platform_name
diff --git a/unittests/test_pipeline.py b/unittests/test_pipeline.py
index 86c4d0036..b45528c78 100644
--- a/unittests/test_pipeline.py
+++ b/unittests/test_pipeline.py
@@ -15,7 +15,6 @@
 import reframe.utility.sanity as sn
 import unittests.utility as test_util
 
-from reframe.core.containers import _STAGEDIR_MOUNT
 from reframe.core.exceptions import (BuildError, PipelineError, ReframeError,
                                      PerformanceError, SanityError,
                                      SkipTestError, ReframeSyntaxError)
@@ -1853,14 +1852,14 @@ def setup_container_platf(self):
 
                 self.container_platform.image = image
                 self.container_platform.command = (
-                    f"bash -c 'cd {_STAGEDIR_MOUNT}; pwd; ls; "
-                    f"cat /etc/os-release'"
+                    "bash -c 'cd /rfm_workdir; pwd; ls; "
+                    "cat /etc/os-release'"
                 )
 
             @sanity_function
             def assert_os_release(self):
                 return sn.all([
-                    sn.assert_found(rf'^{_STAGEDIR_MOUNT}', self.stdout),
+                    sn.assert_found(r'^/rfm_workdir', self.stdout),
                     sn.assert_found(r'^foo', self.stdout),
                     sn.assert_found(
                         r'18\.04\.\d+ LTS \(Bionic Beaver\)', self.stdout),