Repository: terrapower/armi
Pull request #1333 (2023-06-28 22:25:42)

Fix _processIncludes to take StringIO argument
## What is the change?

Added the capability for `_processIncludes` to handle `StringIO` input for `src` argument. 

## Why is the change being made?

`resolveMarkupInclusions` claims to handle `StringIO` and `TextIOBase` as inputs for the `src`, but `_processIncludes` only handled the scenario where there was a file path input. 

This is discussed more in #1329.



---

## Checklist

<!--
    You (the pull requester) should put an `x` in the boxes below you have completed.
    If you're unsure about any of them, don't hesitate to ask. We're here to help!
    Learn what a "good PR" looks like here:
    https://terrapower.github.io/armi/developer/tooling.html#good-pull-requests
-->

- [x] This PR has only one purpose or idea.
- [x] Tests have been added/updated to verify that the new/changed code works.

<!-- Check the code quality -->

- [x] The code style follows [good practices](https://terrapower.github.io/armi/developer/standards_and_practices.html).
- [x] The commit message(s) follow [good practices](https://terrapower.github.io/armi/developer/tooling.html).

<!-- Check the project-level cruft -->

- [x] The [release notes](https://terrapower.github.io/armi/release/index.html) (location `doc/release/0.X.rst`) are up-to-date with any important changes.
- [x] The documentation is still up-to-date in the `doc` folder.
- [x] The dependencies are still up-to-date in `setup.py`.


opotowsky: @john-science Here's the deal... 

`TextIOBase` SHOULD in THEORY have a `readlines` attribute. I just couldn't figure out how to create a `TextIOBase` object in my code properly to test that. Which means the unit test I would add isn't going to work. 

Yet?

BRB will work on this tomorrow :-) 

Patch:
diff --git a/armi/utils/textProcessors.py b/armi/utils/textProcessors.py
index 5108efb7..b498f45b 100644
--- a/armi/utils/textProcessors.py
+++ b/armi/utils/textProcessors.py
@@ -58,10 +58,10 @@ class FileMark:
 
 
 def _processIncludes(
-    src,
+    src: Union[TextIO, pathlib.Path],
     out,
     includes: List[Tuple[pathlib.Path, FileMark]],
-    root,
+    root: pathlib.Path,
     indentation=0,
     currentFile="<stream>",
 ):
@@ -88,7 +88,13 @@ def _processIncludes(
             return 0
 
     indentSpace = " " * indentation
-    for i, line in enumerate(src.readlines()):
+    if hasattr(src, "getvalue"):
+        # assume stringIO
+        lines = [l + "\n" for l in src.getvalue().split("\n")]
+    else:
+        # assume file stream or TextIOBase, and it has a readlines attr
+        lines = src.readlines()
+    for i, line in enumerate(lines):
         leadingSpace = indentSpace if i > 0 else ""
         m = _INCLUDE_RE.match(line)
         if m:
@@ -134,7 +140,7 @@ def resolveMarkupInclusions(
 
     Parameters
     ----------
-    src : TextIOBase or Path
+    src : StringIO or TextIOBase/Path
         If a Path is provided, read text from there. If is stream is provided, consume
         text from the stream. If a stream is provided, ``root`` must also be provided.
     root : Optional Path
diff --git a/doc/release/0.2.rst b/doc/release/0.2.rst
index 22e1ff30..8107f41c 100644
--- a/doc/release/0.2.rst
+++ b/doc/release/0.2.rst
@@ -15,6 +15,7 @@ What's new in ARMI
 
 Bug fixes
 ---------
+#. Fix _processIncludes to handle StringIO input (`PR#1333 <https://github.com/terrapower/armi/pull/1333>`_)
 #. TBD
 
 ARMI v0.2.8


Test patch:
diff --git a/armi/utils/tests/resources/lower/includeA.yaml b/armi/utils/tests/resources/lower/includeA.yaml
index 74ae1e98..39e14f1d 100644
--- a/armi/utils/tests/resources/lower/includeA.yaml
+++ b/armi/utils/tests/resources/lower/includeA.yaml
@@ -1,4 +1,4 @@
 full_name: Jennifer Person
 # some comment in includeA
-children:
-    !include includeB.yaml
+children: 
+  !include includeB.yaml
diff --git a/armi/utils/tests/resources/root.yaml b/armi/utils/tests/resources/root.yaml
index e5b07d37..e37c2dfb 100644
--- a/armi/utils/tests/resources/root.yaml
+++ b/armi/utils/tests/resources/root.yaml
@@ -1,10 +1,10 @@
 # Behold, the Person family
 bobby: &bobby
-    full_name: Robert Person
+  full_name: Robert Person
 
 billy:
-    full_name: William Person
-    # comment
-    children:
-        - *bobby
-        - !include lower/includeA.yaml
+  full_name: William Person
+  # comment
+  children:
+  - *bobby
+  - !include lower/includeA.yaml
diff --git a/armi/utils/tests/test_textProcessors.py b/armi/utils/tests/test_textProcessors.py
index 9df369c9..84667652 100644
--- a/armi/utils/tests/test_textProcessors.py
+++ b/armi/utils/tests/test_textProcessors.py
@@ -12,11 +12,11 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 """Tests for functions in textProcessors.py."""
+from io import StringIO, TextIOWrapper, TextIOBase
 import os
 import pathlib
-import unittest
-
 import ruamel
+import unittest
 
 from armi.utils import textProcessors
 
@@ -60,6 +60,23 @@ class YamlIncludeTest(unittest.TestCase):
         self.assertTrue(commentFound)
         self.assertTrue(anchorFound)
 
+    def test_resolveIncludes_StringIO(self):
+        """Tests that resolveMarkupInclusions handles StringIO input"""
+        yaml = ruamel.yaml.YAML()
+        with open(os.path.join(RES_DIR, "root.yaml")) as f:
+            loadedYaml = yaml.load(f)
+        stringIO = StringIO()
+        yaml.dump(loadedYaml, stringIO)
+        resolved = textProcessors.resolveMarkupInclusions(
+            src=stringIO, root=pathlib.Path(RES_DIR)
+        )
+        with open(os.path.join(RES_DIR, "root.yaml")) as f:
+            expected = textProcessors.resolveMarkupInclusions(
+                f, root=pathlib.Path(RES_DIR)
+            )
+        # strip it because one method gives an extra newline we don't care about
+        self.assertEqual(resolved.getvalue().strip(), expected.getvalue().strip())
+
     def test_findIncludes(self):
         includes = textProcessors.findYamlInclusions(
             pathlib.Path(RES_DIR) / "root.yaml"