Repository: terrapower/armi
Pull request #1917 (2024-09-30 21:30:41)

Loading some ARMI DBs without the App that created them
## What is the change?

This PR add supports for loading of some ARMI Databases without the App that created them. This is not meant to be a "complete solution" to load ANY ARMI database with that app that created it. But it should help ease the burden a bit.

Also, since reading a data file without understanding the program that created it is a potentially hazardous idea, I have placed some training wheels on the resultant reactor data model. The parameters on this data model are set to read-only mode. This is to make it CLEAR that you do not have enough information to continue to time-evolve or alter this data model.

## Why is the change being made?

This was a user request.

---

## Checklist

- [x] This PR has only [one purpose or idea](https://terrapower.github.io/armi/developer/tooling.html#one-idea-one-pr).
- [x] [Tests](https://terrapower.github.io/armi/developer/tooling.html#test-it) have been added/updated to verify any new/changed code.

<!-- Check the code quality -->

- [x] The code style follows [good practices](https://terrapower.github.io/armi/developer/standards_and_practices.html).
- [x] The commit message(s) follow [good practices](https://terrapower.github.io/armi/developer/tooling.html).

<!-- Check the project-level cruft -->

- [x] The [release notes](https://terrapower.github.io/armi/developer/tooling.html#add-release-notes) have been updated if necessary.
- [x] The [documentation](https://terrapower.github.io/armi/developer/tooling.html#document-it) is still up-to-date in the `doc` folder.
- [x] The dependencies are still up-to-date in `pyproject.toml`.

john-science: @alexhjames @jeffbaylor I just want to keep this on everyone's radar. I'm still hoping for a review.
drewj-tp: Passing comment that came to me this morning: this only works if all the materials and shapes in the database are loadable, right? This change seems to
- skip blueprints if we don't know how to process them
- set all parameters to read only

Which is good.

But if my application defines a new component that only exists in my application, we will have no way to load that, correct? Same for a material.

If we really want to make it possible so any application can open any database, as the title here suggests, those are two additional things we'd need to handle
john-science: > But if my application defines a new component that only exists in my application, we will have no way to load that, correct? Same for a material.
> 
> If we really want to make it possible so any application can open any database, as the title here suggests, those are two additional things we'd need to handle

Correct.  There are actually many more things that people can do in their code that will make it hard to open an ARMI DB without the Application that created it.

Please note that the code in this PR does not suggest a complete solution to that problem. I provide a tool to ignore blueprint sections, and another to ignore parameters. That's really all you get here.

The only "complete solution" to load a DB without the App that created it would mean storing huge portions of that code in the DB.  And I don't think that's a rabbit hole we want to go down, it's been tried before.

TLDR; the title of this PR and its ticket are perhaps the problem here. I grant that. Here I provide 2 or 3 tools that downstream users can try to use to solve the problem, but only in a limited fashion.

Patch:
diff --git a/armi/bookkeeping/db/database3.py b/armi/bookkeeping/db/database3.py
index a5056d88..2930cb43 100644
--- a/armi/bookkeeping/db/database3.py
+++ b/armi/bookkeeping/db/database3.py
@@ -74,6 +74,7 @@ from armi.reactor.blocks import Block
 from armi.reactor.components import Component
 from armi.reactor.composites import ArmiObject
 from armi.reactor.parameters import parameterCollections
+from armi.reactor.reactorParameters import makeParametersReadOnly
 from armi.reactor.reactors import Core, Reactor
 from armi.settings.fwSettings.globalSettings import CONF_SORT_REACTOR
 from armi.utils import getNodesPerCycle
@@ -708,13 +709,11 @@ class Database3:
         statePointName=None,
         allowMissing=False,
     ):
-        """Load a new reactor from (cycle, node).
+        """Load a new reactor from a DB at (cycle, node).
 
-        Case settings and blueprints can be provided by the client, or read from the
-        database itself. Providing these from the client could be useful when
-        performing snapshot runs or where it is expected to use results from a run
-        using different settings and continue with new settings (or if blueprints are
-        not on the database). Geometry is read from the database itself.
+        Case settings and blueprints can be provided, or read from the database. Providing these
+        can be useful for snapshot runs or when you want to change settings mid-simulation. Geometry
+        is read from the database.
 
         .. impl:: Users can load a reactor from a DB.
             :id: I_ARMI_DB_R_LOAD
@@ -734,14 +733,13 @@ class Database3:
         cycle : int
             Cycle number
         node : int
-            Time node. If value is negative, will be indexed from EOC backwards
-            like a list.
-        cs : armi.settings.Settings (optional)
+            Time node. If value is negative, will be indexed from EOC backwards like a list.
+        cs : armi.settings.Settings, optional
             If not provided one is read from the database
-        bp : armi.reactor.Blueprints (optional)
+        bp : armi.reactor.Blueprints, optional
             If not provided one is read from the database
-        statePointName : str
-            Optional arbitrary statepoint name (e.g., "special" for "c00n00-special/")
+        statePointName : str, optional
+            Statepoint name (e.g., "special" for "c00n00-special/")
         allowMissing : bool, optional
             Whether to emit a warning, rather than crash if reading a database
             with undefined parameters. Default False.
@@ -798,6 +796,37 @@ class Database3:
 
         return root
 
+    def loadReadOnly(self, cycle, node, statePointName=None):
+        """Load a new reactor, in read-only mode from a DB at (cycle, node).
+
+        Parameters
+        ----------
+        cycle : int
+            Cycle number
+        node : int
+            Time node. If value is negative, will be indexed from EOC backwards like a list.
+        statePointName : str, optional
+            Statepoint name (e.g., "special" for "c00n00-special/")
+
+        Returns
+        -------
+        Reactor
+            The top-level object stored in the database; a Reactor.
+        """
+        r = self.load(cycle, node, statePointName=statePointName, allowMissing=True)
+        self._setParamsBeforeFreezing(r)
+        makeParametersReadOnly(r)
+        return r
+
+    @staticmethod
+    def _setParamsBeforeFreezing(r):
+        """Set some special case parameters before they are made read-only."""
+        for child in r.getChildren(deep=True):
+            if not isinstance(child, Component):
+                continue
+            # calling Component.getVolume() sets the volume parameter
+            child.getVolume()
+
     @staticmethod
     def _assignBlueprintsParams(blueprints, groupedComps):
         for compType, designs in (
diff --git a/armi/bookkeeping/db/passiveDBLoadPlugin.py b/armi/bookkeeping/db/passiveDBLoadPlugin.py
new file mode 100644
index 00000000..4dcb9ee3
--- /dev/null
+++ b/armi/bookkeeping/db/passiveDBLoadPlugin.py
@@ -0,0 +1,107 @@
+# Copyright 2024 TerraPower, LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Provides the ability to ignore parameters sections of blueprint files.
+
+This plugin can allow you to more easily open a database, because you can ignore sections of the
+blueprint files, and ignore any parameters as you want.
+
+This was designed to allow loading an ARMI database without the application that created it.
+"""
+import yamlize
+
+from armi import plugins
+from armi.reactor import parameters
+from armi.reactor.parameters import ParamLocation
+from armi.utils import units
+
+
+class PassThroughYamlize(yamlize.Object):
+    """Just a helper for PassiveDBLoadPlugin, to allow for ignore unknown blueprints sections."""
+
+    @classmethod
+    def from_yaml(cls, loader, node, round_trip_data=None):
+        return yamlize.Object.from_yaml.__func__(
+            PassThroughYamlize, loader, node, round_trip_data
+        )
+
+
+class PassiveDBLoadPlugin(plugins.ArmiPlugin):
+    """Provides the ability to passively load a reactor data model from an ARMI DB even if there are
+    unknown parameters and blueprint sections.
+
+    This plugin allows you two define two things:
+
+    1. Sections of blueprint files to ignore entirely.
+    2. A collection of unknown parameters that will be loaded without units or underlying metadata.
+
+    To use this plugin, you need to set two class variables before instantiating the ARMI App:
+
+    1. Set ``SKIP_BP_SECTIONS`` to a list of BP section names (strings).
+    2. Set ``UNKNOWN_PARAMS`` to a mapping from param class to name: ``{Core: ["a", "b", "c"]}``
+
+    Notes
+    -----
+    Obviously, if you are loading huge numbers of unknown parameters and ignoring whole sections of
+    blueprints, you are losing information. There is no way to use this plugin and still claim full
+    fidelity of your understanding of the reactor. ARMI does not support any such claims.
+    """
+
+    SKIP_BP_SECTIONS = []
+    UNKNOWN_PARAMS = {}
+
+    @staticmethod
+    @plugins.HOOKIMPL
+    def defineBlueprintsSections():
+        """Ignore a pre-determined set of blueprint sections."""
+        skips = []
+        for skippedBp in PassiveDBLoadPlugin.SKIP_BP_SECTIONS:
+            skips.append(
+                (
+                    skippedBp.replace(" ", ""),
+                    yamlize.Attribute(
+                        key=skippedBp, type=PassThroughYamlize, default=None
+                    ),
+                    PassThroughYamlize,
+                )
+            )
+
+        return skips
+
+    @staticmethod
+    @plugins.HOOKIMPL
+    def defineParameters():
+        """Define parameters for the plugin."""
+        # build all the parameters we are missing in default ARMI
+        params = {}
+        for dataClass, paramNames in PassiveDBLoadPlugin.UNKNOWN_PARAMS.items():
+            if len(paramNames):
+                params[dataClass] = PassiveDBLoadPlugin.buildParamColl(paramNames)
+
+        return params
+
+    @staticmethod
+    def buildParamColl(names):
+        """Try replacing any missing parameters with unitless nonsense."""
+        # build a collection of defaulted parameters to passively ignore
+        desc = "This is just a placeholder Parameter; it's meaning is unknown."
+        pDefs = parameters.ParameterDefinitionCollection()
+        with pDefs.createBuilder(location=ParamLocation.AVERAGE) as pb:
+            for param in names:
+                pb.defParam(
+                    param, units=units.UNITLESS, description=desc, saveToDB=False
+                )
+
+        return pDefs
diff --git a/armi/reactor/parameters/parameterCollections.py b/armi/reactor/parameters/parameterCollections.py
index 44cc06d3..e3cc0d88 100644
--- a/armi/reactor/parameters/parameterCollections.py
+++ b/armi/reactor/parameters/parameterCollections.py
@@ -146,6 +146,10 @@ class ParameterCollection(metaclass=_ParameterCollectionType):
             should come from a call to __getstate__(). This should only be used
             internally to this model.
         """
+        # add a hook to make this readOnly
+        self._slots.add("readOnly")
+        self.readOnly = False
+
         if self.pDefs is None or not self.pDefs.locked:
             type(self).applyParameters()
 
@@ -276,6 +280,14 @@ class ParameterCollection(metaclass=_ParameterCollectionType):
             "a ParameterCollection!".format(key)
         )
 
+        if getattr(self, "readOnly", False):
+            if key == "readOnly":
+                raise RuntimeError(
+                    "A read-only Parameter Collection cannot be made writeable."
+                )
+            else:
+                raise RuntimeError(f"Cannot set a read-only parameter {key}.")
+
         object.__setattr__(self, key, value)
 
     def __deepcopy__(self, memo):
diff --git a/armi/reactor/reactorParameters.py b/armi/reactor/reactorParameters.py
index 9b0f0be8..90eeb185 100644
--- a/armi/reactor/reactorParameters.py
+++ b/armi/reactor/reactorParameters.py
@@ -770,3 +770,24 @@ def defineCoreParameters():
         )
 
     return pDefs
+
+
+def makeParametersReadOnly(r):
+    """Convert all the paramters in a Reactor to read-only.
+
+    This method is pretty simple. It goes through all the children of a Reactor object,
+    recursively, and converts the parameters to read-only mode. This will affect the Core, but also
+    any Spent Fuel Pools or other high-level reactor systems.
+
+    Parameters
+    ----------
+    r : Reactor
+        Full reactor object, to be modified.
+
+    Notes
+    -----
+    Once you make one Reactor read-only, you cannot make it writeable again.
+    """
+    r.p.readOnly = True
+    for child in r.getChildren(deep=True):
+        child.p.readOnly = True
diff --git a/doc/release/0.4.rst b/doc/release/0.4.rst
index 768c967b..aa2f1192 100644
--- a/doc/release/0.4.rst
+++ b/doc/release/0.4.rst
@@ -12,7 +12,7 @@ New Features
 #. ARMI now supports Python 3.12. (`PR#1813 <https://github.com/terrapower/armi/pull/1813>`_)
 #. Removing the ``tabulate`` dependency by ingesting it to ``armi.utils.tabulate``. (`PR#1811 <https://github.com/terrapower/armi/pull/1811>`_)
 #. Adding ``--skip-inspection`` flag to ``CompareCases`` CLI. (`PR#1842 <https://github.com/terrapower/armi/pull/1842>`_)
-#. Allow merging a component with zero area into another component (`PR#1858 <https://github.com/terrapower/armi/pull/1858>`_)
+#. Allow merging a component with zero area into another component. (`PR#1858 <https://github.com/terrapower/armi/pull/1858>`_)
 #. Use ``Block.getNumPins()`` in ``HexBlock._rotatePins()``. (`PR#1859 <https://github.com/terrapower/armi/pull/1859>`_)
 #. Provide utilities for determining location of a rotated object in a hexagonal lattice (``getIndexOfRotatedCell``). (`PR#1846 <https://github.com/terrapower/armi/1846>`_)
 #. Allow merging a component with zero area into another component. (`PR#1858 <https://github.com/terrapower/armi/pull/1858>`_)
@@ -39,8 +39,8 @@ API Changes
 #. Removing ``Assembly.doubleResolution()``. (`PR#1951 <https://github.com/terrapower/armi/pull/1951>`_)
 #. Removing ``assemblyLists.py`` and the ``AssemblyList`` class. (`PR#1891 <https://github.com/terrapower/armi/pull/1891>`_)
 #. Removing ``Assembly.rotatePins`` and ``Block.rotatePins``. Prefer ``Assembly.rotate`` and ``Block.rotate``. (`PR#1846 <https://github.com/terrapower/armi/1846>`_)
-#. Transposing ``pinMgFluxes`` parameters so that leading dimension is pin index (`PR#1937 <https://github.com/terrapower/armi/pull/1937>`)
-#. Removing ``globalFluxInterface.DoseResultsMapper`` class (`PR#1952 <https://github.com/terrapower/armi/pull/1952>`)
+#. Transposing ``pinMgFluxes`` parameters so that leading dimension is pin index. (`PR#1937 <https://github.com/terrapower/armi/pull/1937>`_)
+#. Removing ``globalFluxInterface.DoseResultsMapper`` class. (`PR#1952 <https://github.com/terrapower/armi/pull/1952>`_)
 #. Removing setting ``mpiTasksPerNode`` and renaming ``numProcessors`` to ``nTasks``. (`PR#1958 <https://github.com/terrapower/armi/pull/1958>`_)
 #. TBD
 


Test patch:
diff --git a/armi/bookkeeping/db/tests/test_databaseInterface.py b/armi/bookkeeping/db/tests/test_databaseInterface.py
index b1542dfa..fa2115d2 100644
--- a/armi/bookkeeping/db/tests/test_databaseInterface.py
+++ b/armi/bookkeeping/db/tests/test_databaseInterface.py
@@ -456,6 +456,21 @@ class TestDatabaseReading(unittest.TestCase):
         self.assertEqual(len(r.core.circularRingList), 0)
         self.assertEqual(len(r.core.blocksByName), 95)
 
+    def test_loadReadOnly(self):
+        with Database3(self.dbName, "r") as db:
+            r = db.loadReadOnly(0, 0)
+
+            # now show we can no longer edit those parameters
+            with self.assertRaises(RuntimeError):
+                r.core.p.keff = 0.99
+
+            b = r.core.getFirstBlock()
+            with self.assertRaises(RuntimeError):
+                b.p.power = 432.1
+
+            for c in b:
+                self.assertGreater(c.getVolume(), 0)
+
     def test_growToFullCore(self):
         with Database3(self.dbName, "r") as db:
             r = db.load(0, 0, allowMissing=True)
diff --git a/armi/reactor/tests/test_parameters.py b/armi/reactor/tests/test_parameters.py
index 38f47141..0cc68999 100644
--- a/armi/reactor/tests/test_parameters.py
+++ b/armi/reactor/tests/test_parameters.py
@@ -11,11 +11,18 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-"""Tests of the Parameters class."""
+"""Tests for assorted Parameters tools."""
+from glob import glob
+from shutil import copyfile
 import copy
+import os
 import unittest
 
 from armi.reactor import parameters
+from armi.reactor.reactorParameters import makeParametersReadOnly
+from armi.reactor.tests.test_reactors import loadTestReactor
+from armi.tests import TEST_ROOT
+from armi.utils.directoryChangers import TemporaryDirectoryChanger
 
 
 class MockComposite:
@@ -471,7 +478,16 @@ class ParameterTests(unittest.TestCase):
     def test_parameterCollectionsHave__slots__(self):
         """Tests we prevent accidental creation of attributes."""
         self.assertEqual(
-            set(["_hist", "_backup", "assigned", "_p_serialNum", "serialNum"]),
+            set(
+                [
+                    "_hist",
+                    "_backup",
+                    "assigned",
+                    "_p_serialNum",
+                    "serialNum",
+                    "readOnly",
+                ]
+            ),
             set(parameters.ParameterCollection._slots),
         )
 
@@ -587,3 +603,29 @@ class ParamCollectionWhere(unittest.TestCase):
             self.assertTrue(check(p), msg=p)
             names.remove(p.name)
         self.assertFalse(names, msg=f"{names=} should be empty")
+
+
+class TestMakeParametersReadOnly(unittest.TestCase):
+    def test_makeParametersReadOnly(self):
+        with TemporaryDirectoryChanger():
+            # copy test reactor to local
+            yamls = glob(os.path.join(TEST_ROOT, "smallestTestReactor", "*.yaml"))
+            for yamlFile in yamls:
+                copyfile(yamlFile, os.path.basename(yamlFile))
+
+            # load some random test reactor
+            _o, r = loadTestReactor(os.getcwd(), inputFileName="armiRunSmallest.yaml")
+
+            # prove we can edit various params at will
+            r.core.p.keff = 1.01
+            b = r.core.getFirstBlock()
+            b.p.power = 123.4
+
+            makeParametersReadOnly(r)
+
+            # now show we can no longer edit those parameters
+            with self.assertRaises(RuntimeError):
+                r.core.p.keff = 0.99
+
+            with self.assertRaises(RuntimeError):
+                b.p.power = 432.1
diff --git a/armi/tests/test_plugins.py b/armi/tests/test_plugins.py
index b56aa402..3674f436 100644
--- a/armi/tests/test_plugins.py
+++ b/armi/tests/test_plugins.py
@@ -26,9 +26,10 @@ from armi import interfaces
 from armi import plugins
 from armi import settings
 from armi import utils
-from armi.reactor.converters.axialExpansionChanger import AxialExpansionChanger
+from armi.bookkeeping.db.passiveDBLoadPlugin import PassiveDBLoadPlugin
 from armi.physics.neutronics import NeutronicsPlugin
 from armi.reactor.blocks import Block
+from armi.reactor.converters.axialExpansionChanger import AxialExpansionChanger
 from armi.reactor.flags import Flags
 from armi.reactor.tests.test_reactors import loadTestReactor, TEST_ROOT
 
@@ -124,10 +125,31 @@ class TestPluginRegistration(unittest.TestCase):
         self.assertIs(first, AxialExpansionChanger)
         pm.register(SillyAxialPlugin)
         second = pm.hook.getAxialExpansionChanger()
-        # Registering a plugin that implements the hook means we get
-        # that plugin's axial expander
+        # Registering a plugin that implements the hook means we get that plugin's axial expander
         self.assertIs(second, SillyAxialExpansionChanger)
 
+    def test_passiveDBLoadPlugin(self):
+        plug = PassiveDBLoadPlugin()
+
+        # default case
+        bpSections = plug.defineBlueprintsSections()
+        self.assertEqual(len(bpSections), 0)
+        params = plug.defineParameters()
+        self.assertEqual(len(params), 0)
+
+        # non-empty cases
+        PassiveDBLoadPlugin.SKIP_BP_SECTIONS = ["hi", "mom"]
+        PassiveDBLoadPlugin.UNKNOWN_PARAMS = {Block: ["fake1", "fake2"]}
+        bpSections = plug.defineBlueprintsSections()
+        self.assertEqual(len(bpSections), 2)
+        self.assertTrue(type(bpSections[0]), tuple)
+        self.assertEqual(bpSections[0][0], "hi")
+        self.assertTrue(type(bpSections[1]), tuple)
+        self.assertEqual(bpSections[1][0], "mom")
+        params = plug.defineParameters()
+        self.assertEqual(len(params), 1)
+        self.assertIn(Block, params)
+
     def test_beforeReactorConstructionHook(self):
         """Test that plugin hook successfully injects code before reactor initialization.