Repository: ESMCI/cime

Issue:
check_input_data check passed for directory, does not confirm if listing is a file
check_input_data appears only to check for the existence of elements listed in the input_data_list, and not whether or not they are files. For example, check_input_data will pass the following:

Buildconf/cice.input_data_list
grid_file = /work/02441/bcash/stampede2/UFS/ufs_inputdata/
kmt_file = /work/02441/bcash/stampede2/UFS/ufs_inputdata/

The subsequent run will fail, however, because grid_file and kmt_file are required.

If I understand the code correctly (which I probably do not) the issue is that check_input_data.py is using os.path.exists in lines 342 and 358 to perform the existence check, but os.path.exists does not distinguish between files and directories. 


Discussion:
Assigning to @WesCoomber since this should be a pretty easy fix.
@benjamin-cash Hi I am having some trouble recreating this error. I created a new test `./create_test SMS.f19_f19.A`  . Then I wiped away some of the dice.input_data_list and did a `case.submit` expecting a run error. There is no run error because it looks like the case.submit regenerates all of the *.input_data_list files in the case/Buildconf/ directory. Could you provide some more information on the steps of how you are encountering the run error?
@WesCoomber - That doesn't quite sound like the problem I am experiencing. The issue I have is that, for whatever reason,  after ./case.build was run BuildConf/cice.input_data_list had incorrect values for two key files. Specifically, they pointed to directories and not files. 

check_input_data did not produce an error because,  as far as I can tell, it is using os.path.exists and the directory does in fact exist. But the application needed a data file, not a directory, so it failed. Does that make sense?
@benjamin-cash Could you let me know which case is creating incorrect values in BuildConf/cice.input_data_list? I am trying to test a fix but the cases I am using automatically generate working *.input_data_list files when I do a ./case.build and when I do a ./case.submit. So that means I am having trouble testing the fix.

Resolving patch:
diff --git a/scripts/lib/CIME/case/check_input_data.py b/scripts/lib/CIME/case/check_input_data.py
--- a/scripts/lib/CIME/case/check_input_data.py
+++ b/scripts/lib/CIME/case/check_input_data.py
@@ -339,7 +339,7 @@ def check_input_data(case, protocol="svn", address=None, input_data_root=None, d
                         # User pointing to a file outside of input_data_root, we cannot determine
                         # rel_path, and so cannot download the file. If it already exists, we can
                         # proceed
-                        if not os.path.exists(full_path):
+                        if not os.path.isfile(full_path):
                             print("Model {} missing file {} = '{}'".format(model, description, full_path))
                             if download:
                                 logger.warning("    Cannot download file since it lives outside of the input_data_root '{}'".format(input_data_root))
@@ -355,7 +355,7 @@ def check_input_data(case, protocol="svn", address=None, input_data_root=None, d
                         # value and ignore it (perhaps with a warning)
                         isdirectory=rel_path.endswith(os.sep)
 
-                        if ("/" in rel_path and not os.path.exists(full_path) and not full_path.startswith('unknown')):
+                        if ("/" in rel_path and not os.path.isfile(full_path) and not full_path.startswith('unknown')):
                             print("Model {} missing file {} = '{}'".format(model, description, full_path))
                             no_files_missing = False
                             if (download):


Test patch: