Repository: cython/cython

Issue:
Misleading __reduce_cython__ error on import
Whenever you use an `import` instead of a `cimport` compiling will work but you might be graced with an unrelated error message at import time.

Can't create easily a self-contained test: I tried and the code was failing with an expected `ImportError`, however you can reproduce it it with:

```bash
git clone git@github.com:psycopg/psycopg3.git mypyerr
cd mypyerr
git checkout 5647fb04dc6eb3e46e2a7bde490feea5be11f2c1
cat <<HERE | patch -p1
--- a/psycopg3/types/text.pyx
+++ b/psycopg3/types/text.pyx
@@ -1,3 +1,4 @@
+from cpython.unicode import PyUnicode_DecodeAscii
 from cpython.unicode cimport PyUnicode_DecodeUTF8
 
 cdef object load_text_binary(const char *data, size_t length, void *context):
HERE

python setup.py develop
python -c "import psycopg3._psycopg3"
```

Which raises the error:
```
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "psycopg3/pq/pq_cython.pyx", line 34, in init psycopg3._psycopg3
    cdef class PGconn:
AttributeError: type object 'psycopg3._psycopg3.PGconn' has no attribute '__reduce_cython__'
```

I believe many help requests about `__reduce_cython__` are related to this.

Tested with Python 3.6 and Cython 3.0a1.



Discussion:
> I believe many help requests about `__reduce_cython__` are related to this.

I'm not so sure. This ticket is about a programming error, and pretty much all cases I've seen so far seemed to involve pip installations or some kind of build setup problem instead. Specifically, whenever a module reloading attempt is involved, this error can occur.

There's this code these days:
https://github.com/cython/cython/blob/fcb44032968d8893ead5c63c7554d293d370ec1b/Cython/Compiler/ModuleNode.py#L2676-L2689
So we already raise an exception on attempts to import a module twice, and prevent re-initialising the same module instance more than once. I can't say right now why these two cases don't produce a better error message (or actually work) in your case. Investigation welcome.

It could actually be that attempts to re-initialise the module come after first clearing the module dict. That would (obviously) break things, and if that's the case, then we need to find a way to actually rebuild the module dict. Or, more simply, *always* raise an exception on re-initialisation attempts. (Or keep track of full init-clear cycles, or something like that.)
I already ran into this issue two or three times myself, but not yet with one of the Cython 3.0 alpha releases, so no comment about the module loading/reinitialization improvements yet.
I however noticed that those errors always pop up whenever I have unnoticed cyclic imports in my code. The only problem about this error is that it doesn't say anything about it. I started googling what this method __reduce_cython__ actually is, what it does and what problems it might cause, but the actual problem was never mentioned. It took me quite a bit of time to figure it out, and now i'm much smarter in that regard, but if this error also pops up in Cython 3, I can imagine newbies getting stuck for several days with it as well.
@Timtam If you have an example of the code that caused this error then it'd be useful. The example given in this issues had too many external dependencies for me to get it to run so something a bit simpler and easier to diagnose would definitely be good (if it exists)
I don't have an example right now, but I might be able to put one together if I don't forget about it and find the time to do so. I'll comment if I found something out.
Right: I have a simple reproducible 2-file example of this error:

```
# mod1.pyx

if globals():
    # don't know why the if-statement is needed. Otherwise I get an "undefined symbol error"
    raise ImportError

cdef class C:
    cdef int a
```

mod1 can be compiled with `cythonize -if mod1.pyx`

```
# tester.py
try:
    import mod1
except:
    import mod1 # have another go
```

Running `python tester.py` gives:

    AttributeError: type object 'mod1.C' has no attribute '__reduce_cython__'
> "undefined symbol error"

Could be due to dead code removal when the last statement is known to raise, i.e. when there is no success return case.
@da-woods hmm, I tried your example in Py3.[678] with latest master but got the expected (chained) `ImportError` in both cases, not the `AttributeError`.
Ah. I think I was testing it on the 0.29 branch. You're right - it seems OK in master. It's clearly harder to reproduce than I thought
@scoder Move the exception before the class definition and it does break in master though. I've edited the example above

Resolving patch:
diff --git a/runtests.py b/runtests.py
--- a/runtests.py
+++ b/runtests.py
@@ -417,6 +417,7 @@ def get_openmp_compiler_flags(language):
                                            'run.test_raisefrom',
                                            'run.different_package_names',
                                            'run.unicode_imports',  # encoding problems on appveyor in Py2
+                                           'run.reimport_failure',  # reimports don't do anything in Py2
                                            ]),
     (3,): (operator.ge, lambda x: x in ['run.non_future_division',
                                         'compile.extsetslice',


Test patch:
diff --git a/tests/pypy_bugs.txt b/tests/pypy_bugs.txt
--- a/tests/pypy_bugs.txt
+++ b/tests/pypy_bugs.txt
@@ -38,6 +38,7 @@ run.index
 run.pyclass_special_methods
 run.reimport_from_package
 run.reimport_from_subinterpreter
+run.reimport_failure
 pkg.cimportfrom
 embedded
 TestCyCache
diff --git a/tests/run/reimport_failure.srctree b/tests/run/reimport_failure.srctree
new file mode 100644
--- /dev/null
+++ b/tests/run/reimport_failure.srctree
@@ -0,0 +1,38 @@
+# mode: run
+# tag: pep489
+
+"""
+PYTHON setup.py build_ext -i
+PYTHON tester.py
+"""
+
+######## setup.py ########
+
+from Cython.Build.Dependencies import cythonize
+from distutils.core import setup
+
+setup(
+  ext_modules = cythonize("*.pyx"),
+)
+
+
+######## failure.pyx ########
+
+if globals():  # runtime True to confuse dead code removal
+    raise ImportError
+
+cdef class C:
+    cdef int a
+
+
+######## tester.py ########
+
+try:
+    try:
+        import failure  # 1
+    except ImportError:
+        import failure  # 2
+except ImportError:
+    pass
+else:
+    raise RuntimeError("ImportError was not raised on second import!")