Repository: DHI/mikeio
Pull request #448 (2022-10-13 06:39:59)

Pfs quotes
Closing #446 


Patch:
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
index fe9211dd..cbbba133 100644
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -23,4 +23,4 @@ If applicable, add screenshots to help explain your problem.
 
 **System information:**
  - Python version
- - MIKE IO version
+ - MIKE version
diff --git a/docs/pfs.md b/docs/pfs.md
index 3028e2bb..7c0add65 100644
--- a/docs/pfs.md
+++ b/docs/pfs.md
@@ -2,156 +2,37 @@
 
 A PFS file is a text file with a tree structure that contains parameters and settings for MIKE tools and engines. MIKE IO can read, modify and create PFS files. 
 
-## The PFS file
-
 The content of the PFS file is similar to a nested dictionary. The root element is often called the *target*. Some PFS files have multiple root elements. The below sections are called *PFS Sections* which can be nested and contain key-value pairs called *keywords* and *parameters*. 
 
-```
-[TARGET1]
-   keywordA = parameterA
-   [SECTION1]
-      keywordB = parameterB
-      keywordC = parameterC
-      [SECTION2]
-         keywordD = parameterD         
-      EndSect  // SECTION2 
-   EndSect  // SECTION1 
-EndSect  // TARGET1
-
-[TARGET2]
-   keywordE = parameterE 
-   [SECTION3]
-      keywordF = parameterF
-   EndSect  // SECTION3 
-EndSect  // TARGET2
-```
-
-## Read
-
-When a PFS file is read with MIKE IO, a `Pfs` object is created. It will contain one or more `PfsSection` objects - one for each target. The PfsSections will typically contain other PfsSections together with a number of key-value pairs. 
+When a PFS file is read with MIKE IO a Pfs object is created. It will contain one or more PfsSection objects - one for each target. The PfsSections will typically contain other PfsSections together with a number of key-value pairs. The PfsSection object is similar to a dictionary. 
 
-A PFS file is read using `mikeio.read_pfs()` (or equivalently `mikeio.Pfs()`):
 
 ```python
 >>> import mikeio
->>> pfs = mikeio.read_pfs("concat.mzt")
-```
-
-The targets can be accessed by their name (as properties), like this:  
-
-```python
->>> pfs.txconc
-CLSID: TxConc.dll
-TypeName: txconc
-CREATEDTIME: '2020-03-11T15:24:45'
-(...)
-```
-
-Or by the `pfs.data` object (which will be a list of PfsSections if the PFS file contains multiple targets). Each of the targets is a `PfsSection` object consisting of key-value pairs (keyword-parameter) and other PfsSections. 
-
-
-### PfsSection
-
-The PfsSection object is similar to a dictionary. You can loop over its contents with `items()`, `keys()` and `values()` like a dictionary. 
-
-```python
->>> pfs.txconc.keys()
-dict_keys(['CLSID', 'TypeName', 'CREATEDTIME', 'MODIFIEDTIME', 'NOTES', 'Setup'])
-```
-
-You can access a specific parameter with the `get()` method: 
-
-```python
->>> pfs.txconc.get("CLSID")
-'TxConc.dll'
-```
-
-Or as a property with dot-notation---which is prefered in most cases as it is more readable: 
-
-```python
->>> pfs.txconc.CLSID
-'TxConc.dll'
-```
-
-A PfsSection can be converted to a dictionary with the to_dict() method: 
-
-```python
->>> pfs.txconc.Setup.File_1.to_dict()
-{'InputFile': '|.\\tide1.dfs1|', 'Items': 1}
-```
-
-If a PfsSection contains enumerated subsections, they can be converted to a pandas DataFrame with the `to_dataframe()` method: 
-
-```python
->>> pfs.txconc.Setup.to_dataframe(prefix="File_")
-        InputFile  Items
-1  |.\tide1.dfs1|      1
-2  |.\tide2.dfs1|      1
+>>> pfs = mikeio.read_pfs("file.pfs")
+>>> pfs
+TODO
 ```
 
+## Read
 
 
-### Unique or non-unique keywords
-
-Depending on the engine intended for reading the PFS file it may or may not make sense to have multiple identical keywords in the same PfsSection. MIKE 21/3 and the marine tools does *not* support non-unique keywords---if on-unique keywords are present, only the first will be read and the presence is most likely a mistake made by hand-editing the file. In other tools, e.g. MIKE Plot Composer, non-unique keywords are used a lot. How MIKE IO shall deal with non-unique keywords can be specified using the `unique_keywords` argument in the `read_pfs()` method: 
-
-```python
->>> import mikeio
->>> pfs = mikeio.read_pfs("myplot.plt", unique_keywords=False)
-```
-
-If a PfsSection contains non-unique PfsSections or keywords, the repeated key will only appear once and the corresponding value will be a list. 
 
 
 ## Update
 
-The PfsSection object can be modified. Existing values can be changed, new key-value pairs can be added, subsections can added or removed. 
-
-
+The PfsSection object can be modified. Existing values can be changes, new key-value pairs can be added, subsections can added or removed. 
 
 ### Modify existing keyword
 
-It is very simple to modify an existing keyword: 
-
-```python
->>> pfs.txconc.Setup.Name = "new name"
-```
-
 
 ### Add new key-value pair
 
-A new key-value pair can be added, like in a dictionary, in this way: 
-
-```python
->>> pfs.txconc.Setup["NewKeyword"] = 12.0
-```
-
-
-### Add new section as a copy of another section
-
-Often a PfsSection is added using an existing PfsSection as a template. 
-
-```python
->>> s = pfs.txconc.Setup.File_1.copy()
->>> s.InputFile = '|.\tide3.dfs1|'
->>> pfs.txconc.Setup["File_3"] = s
-```
-
-
-
-### Add new section from a dictionary
-
-A PfsSection can be created from a dictionary and then added to another PfsSection like any other key-value pair: 
-
-```python
->>> d = {'InputFile': '|.\\tide4.dfs1|', 'Items': 1}
->>> s = mikeio.PfsSection(d)
->>> pfs.txconc.Setup["File_4"] = s
-```
 
+### Add new section
 
 
-## Write to file
+## Write 
 
 A Pfs object can be written to a PFS file using the `write` method. 
 
diff --git a/mikeio/pfs.py b/mikeio/pfs.py
index 0e873454..4707e900 100644
--- a/mikeio/pfs.py
+++ b/mikeio/pfs.py
@@ -21,7 +21,7 @@ def read_pfs(filename, encoding="cp1252", unique_keywords=True):
     unique_keywords: bool, optional
         Should the keywords in a section be unique? Some tools e.g. the
         MIKE Plot Composer allows non-unique keywords.
-        by default True (issue warnings if non-unique keywords are present and use only first)
+        by default True.
 
     Returns
     -------
@@ -45,12 +45,6 @@ class PfsSection(SimpleNamespace):
         # return json.dumps(self.to_dict(), indent=2)
         return yaml.dump(self.to_dict(), sort_keys=False)
 
-    def __len__(self):
-        return len(self.__dict__)
-
-    def __contains__(self, key):
-        return key in self.keys()
-
     def __getitem__(self, key):
         return getattr(self, key)
 
@@ -78,31 +72,23 @@ class PfsSection(SimpleNamespace):
             self.__setattr__(key, value)
 
     def pop(self, key, *args):
-        """If key is in the dictionary, remove it and return its 
-        value, else return default. If default is not given and 
-        key is not in the dictionary, a KeyError is raised."""
+        # if key in self.keys():
+        #     self.__delattr__(key)
         return self.__dict__.pop(key, *args)
 
     def get(self, key, *args):
-        """Return the value for key if key is in the PfsSection,
-        else default. If default is not given, it defaults to None,
-        so that this method never raises a KeyError."""
         return self.__dict__.get(key, *args)
 
     def clear(self):
-        """Remove all items from the PfsSection."""
         return self.__dict__.clear()
 
     def keys(self):
-        """Return a new view of the PfsSection's keys"""
         return self.__dict__.keys()
 
     def values(self):
-        """Return a new view of the PfsSection's values."""
         return self.__dict__.values()
 
     def items(self):
-        """Return a new view of the PfsSection's items ((key, value) pairs)"""
         return self.__dict__.items()
 
     # TODO: better name
@@ -123,7 +109,6 @@ class PfsSection(SimpleNamespace):
                 self[k] = new_value
 
     def copy(self):
-        """Return a copy of the PfsSection."""
         # is all this necessary???
         d = self.__dict__.copy()
         for key, value in d.items():
@@ -132,26 +117,12 @@ class PfsSection(SimpleNamespace):
         return self.__class__(d)
 
     def to_Pfs(self, name: str):
-        """Convert to a Pfs object (with this PfsSection as the target)
-
-        Parameters
-        ----------
-        name : str
-            Name of the target (=key that refer to this PfsSection)
-
-        Returns
-        -------
-        Pfs
-            A Pfs object
-        """
         return Pfs(self, names=[name])
 
     def to_file(self, filename, name: str) -> None:
-        """Write to a Pfs file (providing a target name)"""
         Pfs(self, names=[name]).write(filename)
 
     def to_dict(self):
-        """Convert to (nested) dict (as a copy)"""
         d = self.__dict__.copy()
         for key, value in d.items():
             if isinstance(value, self.__class__):
@@ -159,18 +130,6 @@ class PfsSection(SimpleNamespace):
         return d
 
     def to_dataframe(self, prefix: str = None) -> pd.DataFrame:
-        """Output enumerated subsections to a DataFrame
-
-        Parameters
-        ----------
-        prefix : str, optional
-            The prefix of the enumerated sections, e.g. "File_", by default None
-
-        Returns
-        -------
-        pd.DataFrame
-            The enumerated subsections as a DataFrame
-        """
         if prefix is not None:
             sections = [
                 k for k in self.keys() if k.startswith(prefix) and k[-1].isdigit()
@@ -283,7 +242,7 @@ class Pfs:
     unique_keywords: bool, optional
         Should the keywords in a section be unique? Some tools e.g. the
         MIKE Plot Composer allows non-unique keywords.
-        by default True (issue warnings if non-unique keywords are present and use only first)
+        by default True.
     """
 
     def __init__(self, input, encoding="cp1252", names=None, unique_keywords=True):
@@ -498,19 +457,20 @@ class Pfs:
 
                 key = s[0:idx]
                 key = key.strip()
-                value = s[(idx + 1) :]
-
-                if (
-                    s[0] == "'" and s[-1] == "'"
-                ):  # This is a quoted string and not a list
-                    s = s
-                else:
-                    if "," in value:
-                        value = f"[{value}]"
+                value = s[(idx + 1) :].strip()
 
                 if key == "start_time":
-                    v = eval(value)
-                    value = datetime(*v)
+                    value = datetime.strptime(value, "%Y, %m, %d, %H, %M, %S").strftime(
+                        "%Y-%m-%d %H:%M:%S"
+                    )
+
+                if len(value) > 2:  # ignore foo = ''
+                    value = value.replace("''", '"')
+
+                if value[0] == "'" and value[-1] == "'" and value.count("'") == 2:
+                    pass  # quoted single string
+                elif "," in value:  # array
+                    value = f"[{value}]"
 
                 s = f"{key}: {value}"
 
@@ -555,6 +515,8 @@ class Pfs:
                 else:
                     v = f"'{v}'"
 
+                v = v.replace('"', "''")
+
         elif isinstance(v, bool):
             v = str(v).lower()  # stick to MIKE lowercase bool notation
 


Test patch:
diff --git a/tests/test_pfs.py b/tests/test_pfs.py
index c80fb5fa..4fc1523b 100644
--- a/tests/test_pfs.py
+++ b/tests/test_pfs.py
@@ -114,15 +114,6 @@ def test_pfssection_copy(d1):
     assert sct3.key1 == 3
     assert sct1.key1 == 2
 
-def test_pfssection_len(d1):
-    sct = mikeio.PfsSection(d1)
-    assert len(sct) == 5
-
-def test_pfssection_contains(d1):
-    sct = mikeio.PfsSection(d1)
-    assert "key1" in sct
-    assert "key2" not in sct
-
 
 def test_pfssection_copy_nested(d1):
     sct1 = mikeio.PfsSection(d1)
@@ -269,29 +260,6 @@ def test_read_write(tmpdir):
     _ = mikeio.Pfs(outfilename)  # try to parse it also
 
 
-def test_read_write_filenames(tmpdir):
-    infilename = "tests/testdata/pfs/filenames.pfs"
-    pfs1 = mikeio.Pfs(infilename)
-    outfilename = os.path.join(tmpdir.dirname, "filenames_out.pfs")
-    pfs1.write(outfilename)
-    assert_txt_files_match(infilename, outfilename)
-    _ = mikeio.Pfs(outfilename)  # try to parse it also
-
-
-def test_read_write_filenames_modified(tmpdir):
-    infilename = "tests/testdata/pfs/filenames.pfs"
-    pfs1 = mikeio.Pfs(infilename)
-    pfs1.FILE_NAMES.file5 = r"..\..\newfile5.dfs0"
-    pfs1.FILE_NAMES.file6 = "|../newfile6.dfs0|"
-    outfilename = os.path.join(tmpdir.dirname, "filenames_out.pfs")
-    pfs1.write(outfilename)
-
-    pfs2 = mikeio.Pfs(outfilename)
-    d1 = pfs1.to_dict()
-    d2 = pfs2.to_dict()
-    assert d1 == d2
-
-
 def test_sw():
 
     pfs = mikeio.Pfs("tests/testdata/pfs/lake.sw")
@@ -546,3 +514,36 @@ EndSect // ENGINE
     assert pfs.ENGINE.fill_list[0] == "foo"
     assert pfs.ENGINE.fill_list[1] == "bar"
     assert pfs.ENGINE.fill_list[2] == "baz"
+
+
+def test_double_single_quotes_in_string(tmpdir):
+    text = """
+[DERIVED_VARIABLE_106]
+            name = 'alfa_PC_T'
+            type = 0
+            dimension = 3
+            description = 'alfa_PC_T, ''light'' adjusted alfa_PC, ugC/gC*m2/uE'
+            EUM_type = 999
+            EUM_unit = 0
+            unit = 'ugC/gC*m2/uE'
+            ID = 597
+EndSect  // DERIVED_VARIABLE_106
+"""
+
+    pfs = mikeio.Pfs(StringIO(text))
+    assert (
+        pfs.DERIVED_VARIABLE_106.description
+        == 'alfa_PC_T, "light" adjusted alfa_PC, ugC/gC*m2/uE'
+    )
+
+    filename = os.path.join(tmpdir, "quotes.pfs")
+
+    pfs.write(filename)
+
+    with open(filename) as f:
+        for line in f:
+            if "description" in line:
+                assert (
+                    line.strip()
+                    == "description = 'alfa_PC_T, ''light'' adjusted alfa_PC, ugC/gC*m2/uE'"
+                )
diff --git a/tests/testdata/pfs/filenames.pfs b/tests/testdata/pfs/filenames.pfs
deleted file mode 100644
index 44b0651f..00000000
--- a/tests/testdata/pfs/filenames.pfs
+++ /dev/null
@@ -1,11 +0,0 @@
-// Created     : 2022-10-07 13:41:29
-// By          : MIKE IO
-// Version     : 1.2.0
-
-[FILE_NAMES]
-   file_1 = './file_1.dfs0'
-   file_2 = '..\..\file_2.dfs0'
-   file_3 = |./file_3.dfs0|
-   file_4 = |..\..\file_4.dfs0|
-EndSect  // FILE_NAMES
-