Repository: ESMCI/cime

Issue:
Permissions should be changed after rpointer files are copied so that write for author is allowed.
We'd like to have files in inputdata to be read-only, so that someone can't accidentally end up changing or removing them. However, as it is now, cime relies on having the rpointer files to have write for author allowed. When cime copies the files for a REFCASE it doesn't change the permission, but cime requires that the author be able to overwrite them when running a case.

Thus the best way around this is when cime copies the rpointer files for a case that it also change the file permissions to open up author write.

@gold2718 @fischer-ncar @cacraigucar @fvitt 


Discussion:
It looks like it's this bit of code in lib/CIME/case/check_input_data.py subroutine stage_refcase.

``` python
        # copy the refcases' rpointer files to the run directory
        for rpointerfile in glob.iglob(os.path.join("{}","*rpointer*").format(refdir)):
            logger.info("Copy rpointer {}".format(rpointerfile))
            safe_copy(rpointerfile, rundir)
        expect(rpointerfile,"Reference case directory {} does not contain any rpointer files".format(refdir))
```

It just needs to also change the write permission in the loop over rpointerfile.
I do not want to hold up the release at this point to fix this without a
lot of extra testing. I think at this point we need to make sure that the
refcases have write permissions until this is fixed after the release.

On Mon, Sep 14, 2020 at 11:17 AM Erik Kluzek <notifications@github.com>
wrote:

> It looks like it's this bit of code in lib/CIME/case/check_input_data.py
> subroutine stage_refcase.
>
>         # copy the refcases' rpointer files to the run directory
>         for rpointerfile in glob.iglob(os.path.join("{}","*rpointer*").format(refdir)):
>             logger.info("Copy rpointer {}".format(rpointerfile))
>             safe_copy(rpointerfile, rundir)
>         expect(rpointerfile,"Reference case directory {} does not contain any rpointer files".format(refdir))
>
> It just needs to also change the write permission in the loop over
> rpointerfile.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/ESMCI/cime/issues/3707#issuecomment-692195545>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB4XCE54T4JPGJHNP3I2G7LSFZF3PANCNFSM4RL3GESA>
> .
>


-- 
Mariana Vertenstein
CESM Software Engineering Group Head
National Center for Atmospheric Research
Boulder, Colorado
Office 303-497-1349
Email: mvertens@ucar.edu

It looks like the change should be to add this line to the loop

`os.chmod(rpointerfile, 0o644)`
OK, so someone will do a PR for this for cime master (and also maint-5.6) sometime after the release.
In email discussion @briandobbins notes another way to handle this "..., would be for CESM at run-time to change the write permission of the file before it's going to write to them. That would need to be done in the fortran code rather than in cime python scripting." I think I prefer to solve this in scripts, but see the advantage of in the fortran code as well.
And, per the email discussion:

Just to clarify, I think I'm in agreement on having files in inputdata being read-only to prevent corruption, but on copying to a user's directory (run or case), they should inherit the permissions of that directory.  Since the run directory will inevitably have user+write permissions, the rpointer files will get user+write then.  (But they won't get group+read, for example, unless the directory itself has group+read.)

So yes, keep inputdata to read-only, but have CIME (not the Fortran code) copy them over with the inherited run-directory / case-directory permissions.  It seems like Bill is indicating the logic for something appropriate already exists, but if that's to give explicit permissions, I recommend inherited permissions.  Though not necessarily for this release, given the time constraints.
Working on this....
Thanks for that clarification @briandobbins. It looks like the reason for the behavior is that safe_copy is being used, which defaults to preserving all metadata including permissions (from the directory it was copied from). So changing arguments on the safe_copy call would be another way to handle it. @billsacks let us know what you think...
I spent some time digging into this. I really thought this must have worked at some point in the past (in part because I periodically run the command to make my own files in inputdata read-only, and I'm pretty sure that I own some refcases). But it seems like the cime behavior in this respect has been the same for at least the last couple of years – i.e., leading to failure if the rpointer file is read-only.

My feeling is that the best solution for now is to either change the `safe_copy` call for the rpointer files to set `preserve_meta=False`, or to do an explicit os.chmod on the rpointer files after copying them to assign them write permission. Either one seems okay to me; I'm not sure which is most conservative (which is my main criteria for choosing one over the other right now, given the upcoming cesm2.2 release). I need to run for an hour, then will see if anyone has expressed opinions when I return.

@mvertens 
An advantage of using os.chmod is that it would work on maint-5.6 as well as recent code. I should also say that, given that this problem seems to have existed for at least a couple of years, we might also choose to wait to fix it until after cesm2.2.0. I will talk with @mvertens in a bit.
I don't think there is much of a difference between the two approaches. And as opposed to @billsacks it looks to me like either one could work in maint-5.6 as well (both branches have a safe_copy call that could have preserve_meta changed).

Another impact of using preserve_meta=False is that the file times will be changed to be the copy time rather than the time from the REFCASE. There's maybe a slight advantage to have the REFCASE time rather than the copy time. But, as such I'd describe the chmod after the fact as being slightly more conservative.
@ekluzek - safe_copy in maint-5.6 has a different implementation that doesn't allow for a preserve_meta argument. (This is part of what led me to think that maybe this was a more recently-introduced bug, but it seems like the behavior is the same on maint-5.6 and master, despite differences in the implementation.)
@mvertens and I talked. Given that this issue seems to exist in maint-5.6 (so CESM2.1) as well as master, it seems important that – for now at least – we change the rpointer permissions back to being owner-writeable. Given how close we are to the CESM2.2 release, and the fact that this hasn't seem to caused any problems with CESM2.1, we'd like to wait to fix this until after CESM2.2.0, so we have time to put any fix through its paces in case the fix causes other unforeseen issues.

So I'm unassigning myself, but I'll take the lead on getting NCAR folks to fix any permissions on cheyenne.

Resolving patch:
diff --git a/scripts/lib/CIME/case/check_input_data.py b/scripts/lib/CIME/case/check_input_data.py
--- a/scripts/lib/CIME/case/check_input_data.py
+++ b/scripts/lib/CIME/case/check_input_data.py
@@ -246,6 +246,7 @@ def stage_refcase(self, input_data_root=None, data_list_dir=None):
         for rpointerfile in glob.iglob(os.path.join("{}","*rpointer*").format(refdir)):
             logger.info("Copy rpointer {}".format(rpointerfile))
             safe_copy(rpointerfile, rundir)
+            os.chmod(os.path.join(rundir, os.path.basename(rpointerfile)), 0o644)
         expect(rpointerfile,"Reference case directory {} does not contain any rpointer files".format(refdir))
         # link everything else
 


Test patch: