Repository: yt-project/yt

Issue:
Convert to using tomli
The library currently depends on the dead "toml" library, instead of the "tomli" library (which was just accepted into the stdlib as "tomllib" for Python 3.11. It should be a really easy swap; it's almost the same API as toml, it just expects binary files instead of unicode ones if using `toml{i,lib}.load`. `loads` is the same. Writing is a separate library.

Working on pyodide at https://github.com/pyodide/pyodide/pull/2234 and don't want to have to port a dead library.



Discussion:
Hi, and welcome to yt!  Thanks for opening your first issue.  We have an issue template that helps us to gather relevant information to help diagnosing and fixing the issue.

@henryiii Thanks for the issue -- and I agree, this is the right thing to do.  Do you happen to know if there is an idiomatic way to handle it for existing installations that may already have `toml` but not `tomli`?
Sadly, yes: https://github.com/pypa/build/blob/5d711b4cb24b2d9ed0884a5a9ba54252440899a2/src/build/__init__.py#L44-L49

Though otherwise the PyPA completely moved over to tomli for pip, setuptools, etc (except for build, we needed to keep it as bootstrappable as possible so that abomination is still in there).

Do you happen know what non-optional requirements in install_requires are actually optional?
(That's going to be extra fun once it has a `if sys.version_info < (3, 11)` toggle around a `tomli`/`tomllib` choice - waiting for a `tomllib` implementation to be available, though)
> Do you happen know what non-optional requirements in install_requires are actually optional?

@henryiii I was about to say "none", but after a more carful check I think `pyyaml` is actually optional. I believe it's the only one though.

I was aware of PEP 680 being accepted and the `toml` package being unsupported, but there was no clear benefit from migrating early. Now that there is, I guess this should be done in time for our next release.

> Do you happen to know if there is an idiomatic way to handle it for existing installations that may already have toml but not tomli?

I'm curious what's the gain we expect here ? Why not just move from a hard dependency to another ?

Resolving patch:
diff --git a/yt/config.py b/yt/config.py
--- a/yt/config.py
+++ b/yt/config.py
@@ -1,7 +1,9 @@
 import os
 import warnings
 
-import toml
+# TODO: import tomllib from the standard library instead in Python >= 3.11
+import tomli as tomllib
+import tomli_w
 from more_itertools import always_iterable
 
 from yt.utilities.configuration_tree import ConfigLeaf, ConfigNode
@@ -144,14 +146,16 @@ def read(self, file_names):
             if not os.path.exists(fname):
                 continue
             metadata = {"source": f"file: {fname}"}
-            self.update(toml.load(fname), metadata=metadata)
+            with open(fname, "rb") as fh:
+                data = tomllib.load(fh)
+            self.update(data, metadata=metadata)
             file_names_read.append(fname)
 
         return file_names_read
 
     def write(self, file_handler):
         value = self.config_root.as_dict()
-        config_as_str = toml.dumps(value)
+        config_as_str = tomli_w.dumps(value)
 
         try:
             # Assuming file_handler has a write attribute
@@ -192,8 +196,8 @@ def _repr_json_(self):
 if not os.path.exists(_global_config_file):
     cfg = {"yt": {}}  # type: ignore
     try:
-        with open(_global_config_file, mode="w") as fd:
-            toml.dump(cfg, fd)
+        with open(_global_config_file, mode="wb") as fd:
+            tomli_w.dump(cfg, fd)
     except OSError:
         warnings.warn("unable to write new config file")
 


Test patch: