Repository: edgedb/edgedb

Issue:
Creating deep nested type gives error `could not serialize result in worker subprocess`
<!-- Please search existing issues to avoid creating duplicates. -->
- EdgeDB Version:
EdgeDB CLI 2.2.0+35980b5
EdgeDB Instance    "version": "2.6+db69671",
- OS Version:
MacOS 12.6 (21G115)

The schema has some types connected by links.
For each link to type `Target` the default is a freshly created as `(insert Target)`.
Most likely, the system can only cope with very short chains of such inserts or the total number of the inserts is too high.

Steps to Reproduce:

1. `edgedb migration create --non-interactive` (takes 2-3 minutes) 
2. `edgedb migrate` (takes a bit more than 2 minutes)
3. echo 'insert `varycon.Henkel.Adhesive Experience`::`ARMasterConfig` { id := <uuid>"dc29dd8b-6a44-47de-9048-a2d58629a193" };' | edgedb

Result:

edgedb error: InternalServerError: could not serialize result in worker subprocess
  Hint: This is most likely a bug in EdgeDB. Please consider opening an issue ticket at https://github.com/edgedb/edgedb/issues/new?template=bug_report.md
  Server traceback:
      Can't pickle local object 'process_insert_body.<locals>.dynamic_get_path':

      Traceback (most recent call last):
        File "/Users/doc/Library/Application Support/edgedb/portable/2.6/lib/python3.10/site-packages/edb/server/compiler_pool/worker_proc.py", line 64, in worker
          pickled = pickle.dumps(data, -1)
      AttributeError: Can't pickle local object 'process_insert_body.<locals>.dynamic_get_path'

<!-- If the issue is about a query error, please also provide your schema -->
Schema:

[default.esdl.zip](https://github.com/edgedb/edgedb/files/10021869/default.esdl.zip)



Discussion:
Yes, this produces a query that is big and deeply nested enough that we overflow python's stack while trying to output the SQL code. 

I'm not sure what your requirements are, but I would definitely recommend *against* this style of schema where you have a default insert for essentially every link.

You can wind up with simple inserts actually doing a number of inserts that is exponential in the number of types. This query here does *473* inserts.

I think this is related to some of the performance woes that you've been reporting in other issues: all of these inserts are actually *big* queries once the defaults are inserted into them, and they take a while to compile. (Which isn't to say that there aren't a lot of migration performance improvements to be made on our side.)
Hi @msullivan thank you for this information!

> I think this is related to some of the performance woes that you've been reporting in other issues: all of these inserts are actually big queries once the defaults are inserted into them, and they take a while to compile.

Yes, that would be a prudent assumption. However, the database was completely empty with regard to data objects when I reported the performance issues. Thusly, there cannot have been any such inserts during migration.

> You can wind up with simple inserts actually doing a number of inserts that is exponential in the number of types. This query here does 473 inserts.

I think this is natural if you put a tree structure into the database. In the SQL era I would have to compute the 473 inserts myself and then execute them. In the post-SQL era I will do ....  the same it seems. ;-)

> Yes, this produces a query that is big and deeply nested enough that we overflow python's stack while trying to output the SQL code.

So, a query with 473 inserts is too big for the system? And it blows the Python (!) stack? I thought EdgeDB was supposed to be a high performance system, as stated on the  [roadmap](https://www.edgedb.com/roadmap) in the Design Principles. 
For high performance we are talking about 10K inserts *per second* at the very least. This is is also rightfully reflected in the EdgeDB showcase on your web page. And I do think that the basic ideas presented on the EdgeDB pages can indeed be implemented with high performant results. But this cannot mean working for 8.896 seconds on a query with 473 inserts and then have a stack overflow. Actually, this cannot mean using Python in any performance critical part of the system probably. 
I have seen some [discussion about this topic](https://github.com/edgedb/edgedb/discussions/3442) where your write:

> Rust can handle this sort of compiler building task pretty well, but you still wind up needing to devote a lot of time to thinking about memory management related issues in a situation where in my opinion you don't really care about them.

I suppose, we are currently looking at a situation where the need to think about the (stack) memory management and other resources would have improved the performance considerably. 

All in all I think (and hope) that I just caught the system on the wrong foot with this type of recursive descent via the `default` values and that this descent is just not performant at the moment. I will write my own query builder for the object creation and then proceed with my tests. There are 1.2 million inserts/updates to come, which will hopefully show a different side of EdgeDB regarding performance. By the way, the file with the 1.2M queries has been generated with Rust fetching from PostgreSQL in 33.94 seconds.




Hi @msullivan,

rereading my post on this issue I feel the need to apologise for my harsh words.
I am sorry if I made you feel bad about this issue. 
As we are seriously pondering to move our companies main asset over to EdgeDB I really want you to succeed with this great project!
Dampening the motivation is the last thing anybody needs, please forgive me!

Thinking about the issue here I have taken a closer look at the EdgeDB source repository. And there I got the impression that the [EdgeQL parser](https://github.com/edgedb/edgedb/tree/master/edb/edgeql-parser/src) already *is* written in Rust.
Would you care to tell me how then the Python stack can comes into play here, please?
Is moving over to Rust maybe work in progress?


I have now implemented inserting the data by generating the 473 inserts myself.
`edgedb` can handle the fetches to generate the data and all inserts within about 2.2 to 3 seconds on my machine.
Considering that there is much more going on under the hood, like setting other default values via selects, I think this is pretty good!

Also the batch import with 1.2 million inserts/updates including thousands of such complex inserts has successfully run through in 25 minutes without any parallelisation.

Simple inserts seem to be about 4 thousand a second without working in parallel. 

To summarise, everything looks really good, and the query generator only seems to have serious performance issues with regard to nested default inserts.

Resolving patch:
diff --git a/edb/pgsql/compiler/__init__.py b/edb/pgsql/compiler/__init__.py
--- a/edb/pgsql/compiler/__init__.py
+++ b/edb/pgsql/compiler/__init__.py
@@ -247,6 +247,11 @@ def run_codegen(
     codegen = pgcodegen.SQLSourceGenerator(pretty=pretty, reordered=reordered)
     try:
         codegen.visit(qtree)
+    except RecursionError:
+        # Don't try to wrap and add context to a recursion error,
+        # since the context might easily be too deeply recursive to
+        # process further down the pipe.
+        raise
     except pgcodegen.SQLSourceGeneratorError as e:  # pragma: no cover
         ctx = pgcodegen.SQLSourceGeneratorContext(
             qtree, codegen.result)


Test patch:
diff --git a/tests/test_edgeql_expressions.py b/tests/test_edgeql_expressions.py
--- a/tests/test_edgeql_expressions.py
+++ b/tests/test_edgeql_expressions.py
@@ -8376,3 +8376,14 @@ async def test_edgeql_no_const_folding_str_concat(self):
             """,
             [R'aaaa\q\n'],
         )
+
+    async def test_edgeql_overflow_error(self):
+        body = 'x+' * 1600 + '0'
+
+        async with self.assertRaisesRegexTx(
+            edgedb.UnsupportedFeatureError,
+            "caused the compiler stack to overflow",
+        ):
+            await self.con.query(f'''
+                with x := 1337, select {body}
+            ''')
diff --git a/tests/test_http_edgeql.py b/tests/test_http_edgeql.py
--- a/tests/test_http_edgeql.py
+++ b/tests/test_http_edgeql.py
@@ -268,6 +268,14 @@ def test_http_edgeql_query_13(self):
             )
         )
 
+    def test_http_edgeql_query_14(self):
+        with self.assertRaisesRegex(
+                edgedb.ConstraintViolationError,
+                r'Minimum allowed value for positive_int_t is 0'):
+            self.edgeql_query(
+                r'''SELECT <positive_int_t>-1''',
+            )
+
     def test_http_edgeql_query_globals_01(self):
         Q = r'''select GlobalTest { gstr, garray, gid, gdef, gdef2 }'''
 
diff --git a/tests/test_http_notebook.py b/tests/test_http_notebook.py
--- a/tests/test_http_notebook.py
+++ b/tests/test_http_notebook.py
@@ -189,7 +189,9 @@ def test_http_notebook_05(self):
                     {
                         'kind': 'error',
                         'error': [
-                            'Error', 'array index 2 is out of bounds', {}
+                            'InvalidValueError',
+                            'array index 2 is out of bounds',
+                            {},
                         ]
                     }
                 ]