Repository: plotly/plotly.py
Pull request: Fix `orjson` loading error
Closes #3567 
Also addresses [plotly/dash#2797](https://github.com/plotly/dash/issues/2797)

## Summary

Address the error described in #3567 by adjusting the module loading logic in `get_module`.

The error can be reproduced by following [these steps](https://github.com/plotly/plotly.py/issues/3567#issuecomment-2025505791).

As best I can tell, the error is caused by a race condition between the multiple threads in the Flask hot reloader, and the fact that Python adds a module to `sys.modules` before the module has been executed [[1]](https://github.com/python/cpython/blob/main/Lib/importlib/_bootstrap.py#L928C9-L935C48) [[2]](https://tenthousandmeters.com/blog/python-behind-the-scenes-11-how-the-python-import-system-works/). If one thread attempts to load `orjson` after it has been added to `sys.modules` but before it has been executed, we'll get the error.

## Fix

- Change logic in `get_module` so that when `should_load=True`, we just call `import_module` right away instead of checking `sys.modules` first. 
- This resolves the issue because `import_module` contains logic to verify that the module is fully initialized
- Logic for when `should_load=False` remains the same: If the module is in `sys.modules` we return it, otherwise we return `None`


## Code PR

- [x] I have read through the [contributing notes](https://github.com/plotly/plotly.py/blob/master/contributing.md) and understand the structure of the package. In particular, if my PR modifies code of `plotly.graph_objects`, my modifications concern the `codegen` files and not generated files.
- [ ] I have added tests (if submitting a new feature or correcting a bug) or
  modified existing tests.
- [ ] For a new feature, I have added documentation examples in an existing or
  new tutorial notebook (please see the doc checklist as well).
- [ ] I have added a CHANGELOG entry if fixing/changing/adding anything substantial.
- [ ] For a new feature or a change in behaviour, I have updated the relevant docstrings in the code to describe the feature or behaviour (please see the doc checklist as well).


Problem statement:
With newer versions of orjson, users need to specify the json engine explicitly (bug?)
Hey

I found out I get an
`AttributeError: partially initialized module 'orjson' has no attribute 'OPT_NON_STR_KEYS'`
if I don't specify this
`plotly.io.json.config.default_engine = 'orjson'`
when using orjson v3.6.6 (latest as of 25jan2022)

Also, additional note for whoever might have this issue: you don't need to uninstall orjson if you don't want to use it. just set the engine to 'json' explicitly. 

I'm using orjson because of the performance claims, although I ran some tests switching between the 2 engines and they seem to yield the same results: using go.Candlestick with 10000 candlesticks and some 4-5 indicators, getting ~0.8sec in each case for creating the plot. My purpose is to improve the dash server performace, but it seems it makes no difference (the web page still renders slower than the ticker even with 600 candles)

Hints from issue discussion:
+1
+1. App is presenting problems after a while. Not sure what's going on.
+1. Happens to me when I use streamlit's `st.plotly_chart` (plotly v5.18.0)
File "/.venv/lib/python3.11/site-packages/plotly/io/_json.py", line 146, in to_json_plotly
opts = orjson.OPT_NON_STR_KEYS | orjson.OPT_SERIALIZE_NUMPY

This line of code may cause this issue
A fix may be using a regular import rather than using importlib to manage this package.
To reproduce using Dash:
1. `pip install orjson`
2. Make a Dash app
3. Trigger reload

If you look at network requests, it'll show this exact error.

Commit message: remove extra orjson load

Patch to review:
diff --git a/packages/python/plotly/_plotly_utils/optional_imports.py b/packages/python/plotly/_plotly_utils/optional_imports.py
index 32dec4e12ab..648626acce5 100644
--- a/packages/python/plotly/_plotly_utils/optional_imports.py
+++ b/packages/python/plotly/_plotly_utils/optional_imports.py
@@ -2,6 +2,7 @@
 Stand-alone module to provide information about whether optional deps exist.
 
 """
+
 from importlib import import_module
 import logging
 import sys
@@ -19,16 +20,18 @@ def get_module(name, should_load=True):
     :return: (module|None) If import succeeds, the module will be returned.
 
     """
-    if name in sys.modules:
-        return sys.modules[name]
     if not should_load:
-        return None
-    if name not in _not_importable:
-        try:
-            return import_module(name)
-        except ImportError:
-            _not_importable.add(name)
-        except Exception:
-            _not_importable.add(name)
-            msg = f"Error importing optional module {name}"
-            logger.exception(msg)
+        return sys.modules.get(name, None)
+
+    else:
+        if name not in _not_importable:
+            try:
+                return import_module(name)
+            except ImportError:
+                _not_importable.add(name)
+            except Exception:
+                _not_importable.add(name)
+                msg = f"Error importing optional module {name}"
+                logger.exception(msg)
+        else:
+            return None


Code review comments:
diff_hunk: @@ -19,16 +20,18 @@ def get_module(name, should_load=True):
     :return: (module|None) If import succeeds, the module will be returned.
 
     """
-    if name in sys.modules:
-        return sys.modules[name]
     if not should_load:
-        return None
-    if name not in _not_importable:
-        try:
-            return import_module(name)
-        except ImportError:
-            _not_importable.add(name)
-        except Exception:
-            _not_importable.add(name)
-            msg = f"Error importing optional module {name}"
-            logger.exception(msg)
+        return sys.modules.get(name, None)
+
+    else:
path: packages/python/plotly/_plotly_utils/optional_imports.py
text: @user1:
The else is not necessary after return.