Repository: edgedb/edgedb

Issue:
Empty set returned by a function with cardinality one, but not returned by an identical computed expression.
- EdgeDB Version: 3.2+44862ed
- EdgeDB CLI Version: 3.4.0+4d95a2c
- OS Version: macOS 13.4.1

Steps to Reproduce:

Setup a database with the below schema.
Run the following query:
```
insert Challenge {
    visibility := "shown"
};
```
Execute the following query (note that `test` is a function identical to the computed's expression):
```
with
  challenge := (select Challenge limit 1),
  user := <User>{},
select {
  func := test(challenge, user),
  comp := (
    (user.is_admin ?? false)
    or (challenge.visibility = "shown")
    or (
      (challenge.visibility = "obfuscated")
      and ((user in challenge.solved_by_users) ?? false)
    )
  )
};
```
I get the following result:
```
{
  "func": null,
  "comp": true
}
```
Given that the expressions in the `test` function and the computed are identical, this doesn't make sense. The `test` function is also has a cardinality that requires a return value.

I have played around with modifying the expression (while keeping the computed and function identical) to find the minimal expression where this issue occurs and have found the following results:

<table>
<tr>
 <td>Expression
 <td>Result
<tr>
 <td>

```
(user.is_admin ?? false)
# or (challenge.visibility = "shown")
or (
  (challenge.visibility = "obfuscated")
  and ((user in challenge.solved_by_users) ?? false)
)
```
 <td>

```
{
  "func": null,
  "comp": false
}
```
<tr>
 <td>

```
# (user.is_admin ?? false)
# or (challenge.visibility = "shown")
# or (
  (challenge.visibility = "obfuscated")
  and ((user in challenge.solved_by_users) ?? false)
# )
```
 <td>

```
{
  "func": false,
  "comp": false
}
```
<tr>
 <td>

```
(user.is_admin ?? false)
# or (challenge.visibility = "shown")
# or (
  # challenge.visibility = "obfuscated"
  # and ((user in challenge.solved_by_users) ?? false)
# )
```
 <td>

```
{
  "func": false,
  "comp": false
}
```
</table>

Schema:
```
module default {
    type User {
        required name: str {
            constraint exclusive;
        }
        required is_admin: bool {
            default := false;
        }
    }

    function test(challenge: Challenge, user: optional User) -> bool
        using (
            (user.is_admin ?? false)
            or (challenge.visibility = "shown")
            or (
                challenge.visibility = "obfuscated"
                and ((user in challenge.solved_by_users) ?? false)
            )
        );

    type Challenge {
        multi link solved_by_users: User;

        required property visibility: str;
    }
}
```


Discussion:
Testing further combinations:

<table>
<tr>
 <td>

```
# (user.is_admin ?? false)
# or
(challenge.visibility = "shown")
or (
  challenge.visibility = "obfuscated"
  and ((user in challenge.solved_by_users) ?? false)
)
```
 <td>

```
{
  "func": true,
  "comp": true
}
```
<tr>
 <td>

```
(user.is_admin ?? false)
or (challenge.visibility = "shown")
# or (
#   challenge.visibility = "obfuscated"
#   and ((user in challenge.solved_by_users) ?? false)
# )
```
 <td>

```
{
  "func": true,
  "comp": true
}
```
</table>

It seems to be the combination of the first and last conditions that cause the mismatch.
I have also tried replacing first condition (`(user.is_admin ?? false)`) with a variety of others (while keeping the final condition the same) and have not yet found another condition that causes this issue. I have tried the following (with notes in brackets about what I was testing):

- `1 + 1 = 2` (always true)
- `1 + 1 = 3` (always false)
- `<bool>{} ?? false` (uses coalesce operator)
- `exists user` (interacts with `user`)

This issue is bizarre.
Here's a reduced version:
```
create function test(user: optional Object) -> str using (
    (<str>user.id ?? "") ++ <str>(exists user)
);
```

```
scope> select test(<Object>{});
{}
```

Elvis spotted that the problem is that we are trying to fetch the User from the database at the top level of the function.

I think something is going wrong with the scoping when an optional argument is used both in an OPTIONAL (`??`) and a SET OF argument (`IN` in the original, `EXISTS` in my minimized version)
I can produce the issue without a function too:
```
edgedb> with user := array_unpack(<array<Object>>[])
....... select
.......     (<str>user.id ?? "") ++ <str>(exists user)
....... ;
{}
```

The issue is the same: when we `ensure_source_rvar`, it gets joined unconditionally, instead of using optional wrapping
It doesn't need to be an optional and a set of; having references in two separate optional arguments fails too:
```
scope> with c := array_unpack(<array<Card>>[])
...... select (c.name ?= c.element);
[]
```

Resolving patch:
diff --git a/edb/pgsql/compiler/relctx.py b/edb/pgsql/compiler/relctx.py
--- a/edb/pgsql/compiler/relctx.py
+++ b/edb/pgsql/compiler/relctx.py
@@ -416,7 +416,6 @@ def new_empty_rvar(
     nullrel = pgast.NullRelation(
         path_id=ir_set.path_id, type_or_ptr_ref=ir_set.typeref)
     rvar = rvar_for_rel(nullrel, ctx=ctx)
-    pathctx.put_rvar_path_bond(rvar, ir_set.path_id)
     return rvar
 
 
diff --git a/edb/pgsql/compiler/relgen.py b/edb/pgsql/compiler/relgen.py
--- a/edb/pgsql/compiler/relgen.py
+++ b/edb/pgsql/compiler/relgen.py
@@ -413,12 +413,51 @@ def _get_set_rvar(
     return process_set_as_root(ir_set, ctx=ctx)
 
 
+def _get_source_rvar(
+    ir_set: irast.Set,
+    scope_stmt: pgast.SelectStmt,
+    *,
+    ctx: context.CompilerContextLevel,
+) -> pgast.PathRangeVar:
+    is_optional = (
+        ctx.scope_tree.is_optional(ir_set.path_id) or
+        ir_set.path_id in ctx.force_optional
+    )
+
+    if not is_optional:
+        rvar = relctx.new_root_rvar(ir_set, lateral=True, ctx=ctx)
+        relctx.include_rvar(
+            scope_stmt, rvar, path_id=ir_set.path_id, ctx=ctx
+        )
+    else:
+        # If the path is optional in the context we are in, then we
+        # need to put optional wrapping around the join with the base table.
+        with ctx.subrel() as subctx:
+            stmt, optrel = prepare_optional_rel(
+                ir_set=ir_set, stmt=subctx.rel, ctx=subctx)
+            subctx.pending_query = subctx.rel = stmt
+
+            rvar = relctx.new_root_rvar(ir_set, lateral=True, ctx=subctx)
+            rvars = new_source_set_rvar(ir_set, rvar)
+            rvars = finalize_optional_rel(
+                ir_set, optrel=optrel, rvars=rvars, ctx=subctx)
+
+            rvar = _include_rvars(rvars, scope_stmt=scope_stmt, ctx=ctx)
+
+    return rvar
+
+
 def ensure_source_rvar(
     ir_set: irast.Set,
     stmt: pgast.Query,
     *,
     ctx: context.CompilerContextLevel,
 ) -> pgast.PathRangeVar:
+    """Make sure that a source aspect is available for ir_set.
+
+    If no aspect is available, compile it. If value/identity is available
+    but source is not, select from the base relation and join it in.
+    """
 
     rvar = relctx.maybe_get_path_rvar(
         stmt, ir_set.path_id, aspect='source', ctx=ctx)
@@ -442,10 +481,7 @@ def ensure_source_rvar(
                 rvar = relctx.get_path_rvar(
                     scope_stmt, ir_set.path_id, aspect='value', ctx=ctx)
             else:
-                rvar = relctx.new_root_rvar(ir_set, lateral=True, ctx=ctx)
-                relctx.include_rvar(
-                    scope_stmt, rvar, path_id=ir_set.path_id, ctx=ctx
-                )
+                rvar = _get_source_rvar(ir_set, scope_stmt, ctx=ctx)
             pathctx.put_path_rvar(stmt, ir_set.path_id, rvar, aspect='source')
 
     return rvar


Test patch:
diff --git a/tests/test_edgeql_coalesce.py b/tests/test_edgeql_coalesce.py
--- a/tests/test_edgeql_coalesce.py
+++ b/tests/test_edgeql_coalesce.py
@@ -1858,3 +1858,35 @@ async def test_edgeql_optional_leakage_01(self):
             ''',
             [],
         )
+
+    async def test_edgeql_optional_ensure_source_01(self):
+        await self.assert_query_result(
+            r'''
+                with x := array_unpack(<array<Issue>>[])
+                select (x.name ?= x.body);
+            ''',
+            [True],
+        )
+
+        await self.assert_query_result(
+            r'''
+                with user := array_unpack(<array<Object>>[])
+                select
+                    (<str>user.id ?? "") ++ <str>(exists user);
+            ''',
+            ["false"],
+        )
+
+    async def test_edgeql_optional_ensure_source_02(self):
+        await self.con.execute('''
+            create function test(x: optional Issue) -> bool using (
+                (x.name ?= x.body)
+            )
+        ''')
+
+        await self.assert_query_result(
+            r'''
+                select test(<Issue>{})
+            ''',
+            [True],
+        )