Repository: ESMCI/cime

Issue:
Add PIO_NETCDF_FORMAT variable 
A recent change to PIO_64BIT_DATA for some file formats has caused some users of older versions of netcdf with unreadable files for post-processing.   This can be fixed by updating to netcdf 4.4.0 or newer but an alternative solution is to add a variable to the PIO namelist 
PIO_NETCDF_FORMAT_XXX  where XXX is the component name.   This would be a string
with allowed values:  'classic', '64bit_offset', '64bit_data', it would be ignored if PIO_TYPENAME was netcdf4c or netcdf4p since that case implies hdf5 format.    


Discussion:
So this would require changes in every component.

As an example in cam - cam_pio_utils.F90 would need the following new statements in cam_pio_createfile: 
```
  
    mode = ior(PIO_CLOBBER, PIO_64BIT_OFFSET)  **(old)**
    mode= ior(PIO_CLOBBER, pio_netcdf_format) **(new)**
    if (present(mode_in)) then
      mode = ior(mode, mode_in)
    end if
    ierr = pio_createfile(pio_subsystem, file, pio_iotype, fname, mode)

```
and in init_pio_subsystem:

```
 subroutine init_pio_subsystem()
    use shr_pio_mod,   only: shr_pio_getiosys, shr_pio_getiotype
    **use shr_pio_mod,  only: shr_pio_netcdf_format (new)**
    use cam_instance, only: atm_id

    **pio_netcdf_format = shr_pio_netcdf_format(atm_id)  (new)**
    pio_subsystem => shr_pio_getiosys(atm_id)
    pio_iotype =  shr_pio_getiotype(atm_id)

    if (masterproc) then
       write(iulog,*)' '
       write(iulog,*)'Initialize PIO subsystem:'
       write(iulog,*)'  iotype  = ', pio_iotype
    end if
  end subroutine init_pio_subsystem
```

But its optional right?  We're updating all our tools to netcdf 4.4 or better.
It is optional and defined in xml - if the components implement the change.
Could you please clarify what that means? I cannot really understand this
sentence. If the components do nothing - what is the behavior they will now
have?

On Wed, Jul 26, 2017 at 1:11 PM, jedwards4b <notifications@github.com>
wrote:

> It is optional and defined in xml - if the components implement the change.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/ESMCI/cime/issues/1760#issuecomment-318153258>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AHlxE7LE7fsVnY-WgUyqpdzsF7bNANlRks5sR49mgaJpZM4OkGTp>
> .
>

Whatever hardcoded behavior they currently have (some have a cdf64 logical that implements pio_64bit_offset), they will need to make the changes in order to use the new variable.  So if cime implements this but the component does not, cime has no way of recognizing that the component is not using it. 
But the component will still be able to use its hardcoded value - correct?
The clarification I'm asking for is that whatever the default behavior was
before our change would be maintained - thereby addressing Mike Levy's
suggestion.



On Wed, Jul 26, 2017 at 1:17 PM, jedwards4b <notifications@github.com>
wrote:

> Whatever hardcoded behavior they currently have (some have a cdf64 logical
> that implements pio_64bit_offset), they will need to make the changes in
> order to use the new variable. So if cime implements this but the component
> does not, cime has no way of recognizing that the component is not using it.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/ESMCI/cime/issues/1760#issuecomment-318154644>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AHlxE2k3hHwb9dwBK2scCd30Fk-3UcSZks5sR5DGgaJpZM4OkGTp>
> .
>


Resolving patch:
diff --git a/scripts/lib/CIME/XML/env_run.py b/scripts/lib/CIME/XML/env_run.py
--- a/scripts/lib/CIME/XML/env_run.py
+++ b/scripts/lib/CIME/XML/env_run.py
@@ -15,10 +15,7 @@ def __init__(self, case_root=None, infile="env_run.xml", components=None):
         """
         self._components = components
         self._component_value_list = ["PIO_TYPENAME", "PIO_STRIDE", "PIO_REARRANGER",
-                                      "PIO_NUMTASKS", "PIO_ROOT"]
+                                      "PIO_NUMTASKS", "PIO_ROOT", "PIO_NETCDF_FORMAT"]
         schema = os.path.join(get_cime_root(), "config", "xml_schemas", "env_entry_id.xsd")
 
         EnvBase.__init__(self, case_root, infile, schema=schema)
-
-
-


Test patch: