Repository: edgedb/edgedb

Issue:
Session cannot change within a Transaction (at least AP enforcement)
<!-- Please search existing issues to avoid creating duplicates. -->

<!--
For the EdgeDB Version: run `edgedb query 'select sys::get_version_as_str()'` from your project directory (or run `select sys::get_version_as_str();` in the EdgeDB interactive shell).
For the EdgeDB CLI Version: Run `edgedb --version` from anywhere
-->

- EdgeDB Version: 4.7+45e4acc
- EdgeDB CLI Version: 4.1
- OS Version: mac 14.4

Minimal reproduction repo:
https://github.com/CarsonF/edgedb-bug-ap-config-ignored

Essentially:
1. Start transaction with non default session, with APs enabled
2. Run query inside transaction with a different session, with APs disabled
   Query is executed with APs enabled instead


<details>
<summary>Original Post</summary>

This is very hard to reproduce. I've spent hours narrowing it down.

Query
```edgeql
WITH __param__email := <std::str>$email
SELECT (
  SELECT std::assert_single(
    (
      WITH __scope_0_AuthIdentity := DETACHED Auth::Identity
      SELECT __scope_0_AuthIdentity { id }
      FILTER (__scope_0_AuthIdentity.user.email = __param__email)
    )
  ).passwordHash
)
```
```ts
const client = createClient().withConfig({
  apply_access_policies: false,
});

client.query("...") // fine
client.querySingle("...") // error
```
```
CardinalityViolationError: required link 'user' of object type 'Auth::Identity' is hidden by access policy
```

It appears that nothing about this query can change, and still reproduce the error.
I've tried:
- inlining email param
- removing `{ id }` shape from Identity
- not aliasing a detached Identity

All work fine. And querying with a `Many` cardinality works, but `AtMostOne` does not.

I'm starting to think that maybe the AP checks are baked into the query which is then cached, and then that cached query is used even when called with APs disabled.

</details>


Discussion:
Ok I just restarted my instance, and that query started working. So it's definitely the query cache not accounting for changes in AP config.
I've never been able to reproduce this in the UI actually. Even with the same query which failed in server, would pass in the UI (no db restarts).

So there must be something to do with the `AtMostOne` cardinality, which the UI never uses.
That's a pretty concerning issue!

Have you been able to get it to happen again after doing the restart? Obviously consistent instructions to reproduce would be hugely helpful, but I understand if it's not possible with an issue like this.

Configuration keys like this one *are* part of the query cache key, so something must be thwarting that mechanism
Definitely. I've spend many hours on red herrings. Knowing that caching is at play helps, and not my other guesses (something with my code, or transactions, or other config options).

I was toggling this config option globally as well (`edgedb configure set ...`).
Maybe between that and the connection's setting something is happening.

I can continue digging on Monday.
FYI I'm also trying to reproduce with the following mocked schema given your query:

```esdl
module default {
  global current_user: uuid;

  type User {
    email: str;
    access policy full
      allow all
      using (global current_user ?= .id);
  }

  type Identity {
    required user: User;
    passwordHash: str;
  }
}
```

No luck so far, but I'll trace into the cache keys and see where it could go wrong.

> I was toggling this config option globally as well (edgedb configure set ...).
> Maybe between that and the connection's setting something is happening.

I'll try this too
Ok I was able to reproduce @fantix 

1. restart db (or fresh)
2. run query with APs enabled globally & connection using default
3. run query with APs disabled globally & connection using default
4. run query with APs enabled globally & connection explicitly disabling - error

Actually now `edgedb instance restart` isn't fixing the error.
I just tried as well with a simpler query, not the query builder generated one, and my steps above triggered the error.
```edgeql
with identity := (select Auth::Identity filter .user.email = <str>$email)
select identity.passwordHash
```
> I just tried as well with a simpler query, not the query builder generated one, and my steps above triggered the error.
> 
> ```edgeql
> with identity := (select Auth::Identity filter .user.email = <str>$email)
> select identity.passwordHash
> ```

This is not running with `client.querySingle()`?

And for the steps you shared above, is the steps 2-4 using the same connection, or it's a script like:

```js
import edgedb from 'edgedb';

const client = edgedb.createClient();
// const client = edgedb.createClient().withConfig({ apply_access_policies: false, });

client.query(`
with identity := (select Identity filter .user.email = <str>$email)
select identity.passwordHash
`, {email: 'test'});
```

And for each step you run it separately after the global config like:

```console
$ edgedb configure set apply_access_policies true  # or false
```

I'm asking because the above is not reproducing the issue for me yet, perhaps I'm missing some details here.
It is running with `querySingle`.
My schema has `User.email` exclusive, and `Identity.user` exclusive.

Correct on global cli command.

Uhh I've used the same connection yes, but I believe it still errors after my web server process restart as well.
So both same connection & different ones.

Could you pls share your schema? Like a minimal part of it to reproduce
[Here's](https://github.com/SeedCompany/cord-api-v3/tree/develop/dbschema) the full schema.
I can try to pear it down, but maybe the size/complexity of it is part of the condition needed to reproduce.
I also tried to create a minimal repo from scratch and was unable to reproduce.

Note that the SDL APs are not versioned (they are in the migrations), but can be generated with
```
yarn edgedb:gen && yarn console edgedb ap inject
```
Oh I believe it's important that the Identity have a User linked, even though the link is optional
Ok sorry new info. The query has to be ran within a transaction.

I stopped my webserver before running the global configures, so it doesn't appear to need to be a single connection.
No prob! I'll look into this asap.
Alright actually I've come full circle here. I think the problem is that config option cannot change **after** the transaction is started.
The JS driver doesn't normally allow this.

I believe I was thrown off this because _globals_ can change with each _query_ within the _transaction_; even though the JS driver doesn't normally allow this. And the _session_ as a whole is sent with every _query_.

I'm willing to accept this that this is on me, and this is expected functionality.
There's no way the `apply_access_policies` would/could change (in future) within each query of a transaction, is there?
Oh wow I think I reproduced, one sec
I swore I had transactions disabled at one point last week and was still hitting it.
TL;DR: I think we're fine, though we'll review the JS binding for this special case.

> Alright actually I've come full circle here. I think the problem is that config option cannot change **after** the transaction is started. The JS driver doesn't normally allow this.

Right, I reproduced by:

1. Run the query with AP enabled. The compilation succeeds, the server caches the SQL, and the query fails as expected at execution. ✅ 
2. Start a transaction normally. ✅
3. Turn off AP manually in tx (I hacked edgedb-js to allow running `configure session ...`). The server tx state is properly updated to have AP off, and sends back the updated session state (AP off) as expected. ✅
4. Run the query within the same tx, and **edgedb-js sends the query with NULL_TYPE_ID for session state**, leading the server resetting session state to default/empty, and picked up the "wrong" cached SQL, ending up with the same execution error. ❌ 

> I believe I was thrown off this because _globals_ can change with each _query_ within the _transaction_; even though the JS driver doesn't normally allow this. And the _session_ as a whole is sent with every _query_.

Just looking at the server code, I'm confused about how globals worked as you described. 🤔

> this is expected functionality

On the protocol level, I believe it's fine if the state is changed inside a transaction; it's really up to the clients which state to send with each query. For the JS client, I'm not sure why it's not echoing the server state type ID in transactions. Maybe we'll change this after further investigations, or not.

> There's no way the `apply_access_policies` would/could change (in future) within each query of a transaction, is there?

Yes on client-binding API level, we'll keep the current `client.withConfig(...)` style.

> I swore I had transactions disabled at one point last week and was still hitting it.

Perhaps the JS client cleared your "custom" state in a similar way.
Thanks for diving in with me @fantix.


> On the protocol level, I believe it's fine if the state is changed inside a transaction; it's really up to the clients which state to send with each query.

Are you saying that the protocol/server _should_ allow for APs to be toggled within individual queries of a transaction?


> For the JS client, I'm not sure why it's not echoing the server state type ID in transactions.

The session/state is sent with each query regardless of transactions. But in my case I found that the `start transaction` query had APs enabled, but the actual query used a different session that had them disabled. Leading to my question above.

> Are you saying that the protocol/server should allow for APs to be toggled within individual queries of a transaction?

@fantix confirmed with me on discord, that this should be allowed.

I'll work on a minimal reproduction of this.
yeah sorry for the late reply
Here we go https://github.com/CarsonF/edgedb-bug-ap-config-ignored

Resolving patch:
diff --git a/edb/server/compiler/compiler.py b/edb/server/compiler/compiler.py
--- a/edb/server/compiler/compiler.py
+++ b/edb/server/compiler/compiler.py
@@ -956,6 +956,18 @@ def compile_in_tx_request(
         )
         request.deserialize(serialized_request, original_query)
 
+        # Apply session differences if any
+        if (
+            request.modaliases is not None
+            and state.current_tx().get_modaliases() != request.modaliases
+        ):
+            state.current_tx().update_modaliases(request.modaliases)
+        if (
+            (session_config := request.session_config) is not None
+            and state.current_tx().get_session_config() != session_config
+        ):
+            state.current_tx().update_session_config(session_config)
+
         units, cstate = self.compile_in_tx(
             state,
             txid,


Test patch:
diff --git a/tests/test_protocol.py b/tests/test_protocol.py
--- a/tests/test_protocol.py
+++ b/tests/test_protocol.py
@@ -679,6 +679,54 @@ async def test_proto_parse_execute_transaction_id(self):
             transaction_state=protocol.TransactionState.NOT_IN_TRANSACTION,
         )
 
+    async def test_proto_state_change_in_tx(self):
+        await self.con.connect()
+
+        # Fixture
+        await self._execute('CREATE MODULE TestStateChangeInTx')
+        await self.con.recv_match(protocol.CommandComplete)
+        await self.con.recv_match(protocol.ReadyForCommand)
+        await self._execute('CREATE TYPE TestStateChangeInTx::StateChangeInTx')
+        await self.con.recv_match(
+            protocol.CommandComplete,
+            status='CREATE TYPE'
+        )
+        await self.con.recv_match(protocol.ReadyForCommand)
+
+        # Collect states
+        await self._execute('''
+            SET MODULE TestStateChangeInTx
+        ''')
+        cc = await self.con.recv_match(protocol.CommandComplete)
+        await self.con.recv_match(protocol.ReadyForCommand)
+        await self._execute('''
+            CONFIGURE SESSION SET allow_user_specified_id := true
+        ''', cc=cc)
+        cc_true = await self.con.recv_match(protocol.CommandComplete)
+        await self.con.recv_match(protocol.ReadyForCommand)
+        await self._execute('''
+            CONFIGURE SESSION SET allow_user_specified_id := false
+        ''')
+        cc_false = await self.con.recv_match(protocol.CommandComplete)
+        await self.con.recv_match(protocol.ReadyForCommand)
+
+        # Start a transaction that doesn't allow_user_specified_id
+        await self._execute('START TRANSACTION', cc=cc_false)
+        await self.con.recv_match(protocol.CommandComplete)
+        await self.con.recv_match(protocol.ReadyForCommand)
+
+        # But insert with session that does allow_user_specified_id
+        await self._execute('''
+            INSERT StateChangeInTx {
+                id := <uuid>'a768e9d5-d908-4072-b370-865b450216ff'
+            };
+        ''', cc=cc_true)
+        await self.con.recv_match(
+            protocol.CommandComplete,
+            status='INSERT'
+        )
+        await self.con.recv_match(protocol.ReadyForCommand)
+
 
 class TestServerCancellation(tb.TestCase):
     @contextlib.asynccontextmanager