Repository: gerlero/foamlib

Issue:
Support for reading files from memory
Hi @gerlero,

As discussed, a really nice feature would be if we could pass data straight from memory to foamlib.

The way I imagine this working is that a file is read like this and the file_content can be passed directly.
```
        with open(file_path, 'rb') as file:
            file_content = file.read()
```

Example workflow for this would be:
1. Read OF file from S3 into memory / buffer
2. Pass content to foamlib directly, without creating a file on disk 
3. Get parsed result, apply any other postprocessing
4. Write back to S3


This would allow us to use something like AWS lambda, which doesn't come with enough disk space to write the files first, for compute.

Many thanks!

Hints from issue comments:
@gerlero: Thanks. I'm thinking API could be something like:

```python
def loads(s: bytes | str, *, include_header: bool = False) ->  foamlib._files._types._File |  foamlib._files._types._Data: ...

def dumps(data: foamlib._files._types._File | foamlib._files._types._Data) -> bytes: ... 
```
@cjbraley: Looks good to me. Key would be that your interaction with FoamFile / FomeFieldFile looks as close as possible to the standard approach.@gerlero: @cjbraley As of `0.9.2`, the `FoamFile` class has static methods `loads` and `dumps` that allow for (respectively) parsing and serializing  FoamFile contents without using `FoamFile` objects and/or requiring access to the filesystem. I've also added a few tests, but let me know if anything doesn't work.@cjbraley: Thanks @gerlero. Really nice and working as intended.

It's a small thing but I do wonder a little about whether, as an additional enhancement, it makes sense for FoamFile be able to accept `bytes` **or** `string` / path. If you pass bytes, you would basically already have the contents variable below.

```
        if not self.__defer_io:
            try:
                contents = self.path.read_bytes()
```

Feels like it would tighten up the workflow a little for users. Just a suggestion, I'll leave it up to you.@gerlero: @cjbraley You're welcome.

The central premise of `foamlib` is that both main classes (i.e., `FoamCase` and `FoamFile`) are [path-like objects](https://docs.python.org/3/glossary.html#term-path-like-object) pointing to the corresponding directory or file (respectively) and allow for in-place inspection and mutation of the underlying data in the filesystem. Given this, the possibility of a `FoamFile` object that does not represent an actual file doesn't feel right to me.@cjbraley: Thanks @gerlero,

I see. Yes, if you want to do in-place mutation that makes sense. What i suggested makes more sense in a read -> mutate -> write pattern.

Solution patch:
diff --git a/foamlib/_files/_files.py b/foamlib/_files/_files.py
index b96428e..ffe4f5d 100644
--- a/foamlib/_files/_files.py
+++ b/foamlib/_files/_files.py
@@ -20,6 +20,7 @@ from ._io import FoamFileIO
 from ._serialization import dumps, normalize_data, normalize_keyword
 from ._types import (
     Data,
+    DataLike,
     Dict_,
     Dimensioned,
     DimensionSet,
@@ -444,6 +445,72 @@ class FoamFile(
             d.pop("FoamFile", None)
         return deepcopy(d)
 
+    @staticmethod
+    def dumps(file: File | DataLike, *, ensure_header: bool = True) -> bytes:
+        """
+        Serialize Python objects to the OpenFOAM format.
+
+        :param file: The Python object to serialize. This can be a dictionary, list,
+            or any other object that can be serialized to the OpenFOAM format.
+        :param ensure_header: Whether to include the "FoamFile" header in the output.
+            If `True`, a header will be included if it is not already present in the
+            input object.
+        """
+        if isinstance(file, Mapping):
+            header = file.get("FoamFile", None)
+            assert isinstance(header, FoamFile.SubDict) or header is None
+            entries: list[bytes] = []
+            for k, v in file.items():
+                if k is not None:
+                    entries.append(
+                        dumps((k, v), keywords=(), header=header, is_item=True)
+                    )
+                else:
+                    entries.append(dumps(v, keywords=(), header=header, is_item=True))
+            ret = b" ".join(entries)
+        else:
+            header = None
+            ret = dumps(file)
+
+        if header is None and ensure_header:
+            class_ = "dictionary"
+            if isinstance(file, Mapping) and "internalField" in file:
+                try:
+                    shape = np.shape(file["internalField"])  # type: ignore [arg-type]
+                except ValueError:
+                    pass
+                else:
+                    if not shape:
+                        class_ = "volScalarField"
+                    elif shape == (3,):
+                        class_ = "volVectorField"
+                    elif shape == (6,):
+                        class_ = "volSymmTensorField"
+                    elif shape == (9,):
+                        class_ = "volTensorField"
+                    elif len(shape) == 1:
+                        class_ = "volScalarField"
+                    elif len(shape) == 2:
+                        if shape[1] == 3:
+                            class_ = "volVectorField"
+                        elif shape[1] == 6:
+                            class_ = "volSymmTensorField"
+                        elif shape[1] == 9:
+                            class_ = "volTensorField"
+
+            header = {"version": 2.0, "format": "ascii", "class": class_}
+
+            ret = (
+                dumps(
+                    {"FoamFile": header},
+                    keywords=(),
+                )
+                + b" "
+                + ret
+            )
+
+        return ret
+
 
 class FoamFieldFile(FoamFile):
     """
diff --git a/foamlib/_files/_serialization.py b/foamlib/_files/_serialization.py
index c6a2c24..19fce59 100644
--- a/foamlib/_files/_serialization.py
+++ b/foamlib/_files/_serialization.py
@@ -136,26 +136,33 @@ def dumps(
     *,
     keywords: tuple[str, ...] | None = None,
     header: Mapping[str, Entry] | None = None,
+    is_item: bool = False,
 ) -> bytes:
+    data = normalize_data(data, keywords=keywords)
+
     if isinstance(data, Mapping):
         return (
             b"{"
             + b" ".join(
                 dumps(
                     (k, v),
-                    keywords=(*keywords, k) if keywords is not None else None,
+                    keywords=keywords,
+                    is_item=True,
                 )
                 for k, v in data.items()
             )
             + b"}"
         )
 
-    data = normalize_data(data, keywords=keywords)
-
-    if keywords is None and isinstance(data, tuple) and len(data) == 2:
+    if is_item and isinstance(data, tuple) and len(data) == 2:
         k, v = data
         ret = dumps(k)
-        val = dumps(v)
+        val = dumps(
+            v,
+            keywords=(*keywords, k)
+            if keywords is not None and isinstance(k, str)
+            else None,
+        )
         if val:
             ret += b" " + val
         if not isinstance(v, Mapping):
@@ -185,7 +192,7 @@ def dumps(
 
         shape = np.shape(data)
         if not shape or (not scalar and shape in ((3,), (6,), (9,))):
-            return b"uniform " + dumps(data)
+            return b"uniform " + dumps(data, is_item=True)
 
         assert isinstance(data, np.ndarray)
         ndim = np.ndim(data)
@@ -201,36 +208,46 @@ def dumps(
             elif shape[1] == 9:
                 tensor_kind = b"tensor"
             else:
-                return dumps(data)
+                return dumps(data, is_item=True)
 
         else:
             return dumps(data)
 
         binary = (header.get("format", "") if header else "") == "binary"
 
-        contents = b"(" + data.tobytes() + b")" if binary else dumps(data)
+        contents = b"(" + data.tobytes() + b")" if binary else dumps(data, is_item=True)
 
-        return b"nonuniform List<" + tensor_kind + b"> " + dumps(len(data)) + contents
+        return (
+            b"nonuniform List<"
+            + tensor_kind
+            + b"> "
+            + dumps(len(data), is_item=True)
+            + contents
+        )
 
     if isinstance(data, DimensionSet):
-        return b"[" + b" ".join(dumps(v) for v in data) + b"]"
+        return b"[" + b" ".join(dumps(v, is_item=True) for v in data) + b"]"
 
     if isinstance(data, Dimensioned):
         if data.name is not None:
             return (
-                dumps(data.name)
+                dumps(data.name, is_item=True)
                 + b" "
-                + dumps(data.dimensions)
+                + dumps(data.dimensions, is_item=True)
                 + b" "
-                + dumps(data.value)
+                + dumps(data.value, is_item=True)
             )
-        return dumps(data.dimensions) + b" " + dumps(data.value)
+        return (
+            dumps(data.dimensions, is_item=True)
+            + b" "
+            + dumps(data.value, is_item=True)
+        )
 
     if isinstance(data, tuple):
-        return b" ".join(dumps(v) for v in data)
+        return b" ".join(dumps(v, is_item=True) for v in data)
 
     if is_sequence(data) and not isinstance(data, tuple):
-        return b"(" + b" ".join(dumps(v) for v in data) + b")"
+        return b"(" + b" ".join(dumps(v, is_item=True) for v in data) + b")"
 
     if data is True:
         return b"yes"


Test patch:
diff --git a/tests/test_files/test_dumps.py b/tests/test_files/test_dumps.py
index f1f469c..a704470 100644
--- a/tests/test_files/test_dumps.py
+++ b/tests/test_files/test_dumps.py
@@ -95,3 +95,19 @@ def test_serialize_data() -> None:
     assert dumps([{"a": "b", "c": "d"}, {"e": "g"}]) == b"({a b; c d;} {e g;})"
     assert dumps(["water", "oil", "mercury", "air"]) == b"(water oil mercury air)"
     assert dumps("div(phi,U)") == b"div(phi,U)"
+
+
+def test_serialize_file() -> None:
+    assert FoamFile.dumps(1.0, ensure_header=False) == b"1.0"
+    assert (
+        FoamFile.dumps(1.0)
+        == b"{FoamFile {version 2.0; format ascii; class dictionary;}} 1.0"
+    )
+    assert (
+        FoamFile.dumps([{"a": "b", "c": "d"}, {"e": "g"}])
+        == b"{FoamFile {version 2.0; format ascii; class dictionary;}} ({a b; c d;} {e g;})"
+    )
+    assert (
+        FoamFile.dumps({"internalField": [[1, 2, 3], [4, 5, 6]]})
+        == b"{FoamFile {version 2.0; format ascii; class volVectorField;}} internalField nonuniform List<vector> 2((1.0 2.0 3.0) (4.0 5.0 6.0));"
+    )