Repository: hyperledger/aries-cloudagent-python

Issue:
Multiple issues in the 0.7.3 release
I'm using this ticket to collect issues that I found in the 0.7.3 release.

**1. Tails file upload fails when a credDef is created and multi ledger support is enabled.**

See discussion on the chat: https://chat.hyperledger.org/channel/aries-cloudagent-python?msg=XodRcp7dFxmncAWma I still do not get this to work.

**2. Revocation notification has a wrong revocation id for V1 exchanges (V2 works as expected)**

For example the credential acked event has the following link to the revocation registry:

```json
{
        "rev_reg_id": "7D2jFxSPJP4zTAP9WAcJWo:4:7D2jFxSPJP4zTAP9WAcJWo:3:CL:571:CustomerBankRevoc:CL_ACCUM:01f6b385-28bc-420a-bcc7-00ac282b1d3c",
        "cred_rev_id": "5"
}
```
But in the revocation event the value of the revocation id becomes 6 instead of 5. 

```json
{
    "thread_id": "indy::7D2jFxSPJP4zTAP9WAcJWo:4:7D2jFxSPJP4zTAP9WAcJWo:3:CL:571:CustomerBankRevoc:CL_ACCUM:01f6b385-28bc-420a-bcc7-00ac282b1d3c::6",
    "comment": null
}
```

**3a. V2 Credential exchange ignores the _auto-respond-credential-request: false_ flag**

If aca-py receives a V2 credential request it immediately responds even if the auto flag is set to false. Works as expected for V1.

**3b. V2 Presentation exchange ignores the _auto-respond-presentation-proposal: true_ flag**

If aca-py receives a V2 presentation proposal and the auto respond flag is set to true aca-py does not automatically respond. Works a expected for V1.

**4. Wallet type askar has issues when receiving V1 credentials.**

Probably same as or related to #1588 Any other time answering a credential offer with a request fails with:

```
2022-01-13 11:40:34,184 aries_cloudagent.admin.server ERROR Handler error with exception: Record not found.
2022-01-13 11:40:34,184 aries_askar.native.aries_askar.ffi.store INFO 	src/ffi/store.rs:1007 | Closed session SessionHandle(980)
2022-01-13 11:40:34,185 aries_askar.native.aries_askar.backend.db_utils INFO 	src/backend/db_utils.rs:168 | Dropped pool connection
Traceback (most recent call last):
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 997, in credential_exchange_send_request
    cred_ex_record, connection_record.my_did
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/manager.py", line 471, in create_request
    await cred_ex_record.save(session, reason="create credential request")
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/messaging/models/base_record.py", line 354, in save
    await storage.update_record(record, record.value, record.tags)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/storage/askar.py", line 121, in update_record
    raise StorageNotFoundError("Record not found") from None
aries_cloudagent.storage.error.StorageNotFoundError: Record not found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 163, in ready_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 200, in debug_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp_apispec/middlewares.py", line 45, in validation_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 301, in check_token
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 391, in setup_context
    return await task
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 1017, in credential_exchange_send_request
    outbound_handler,
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/__init__.py", line 63, in report_problem
    raise http_error_class(reason=err.roll_up) from err
aiohttp.web_exceptions.HTTPBadRequest: Record not found.
```


Discussion:
@shaangill025 and @andrewwhitehead --- can the two of you please look at these?  Shaanjot likely the 1, 2 and 3 as they are in the same area, Andrew the fourth because of the Askar connection.

Thanks
For #4, it appears that the process is failing when trying to update the credential exchange record because it doesn't exist. This could be because auto-respond was enabled and the exchange has already completed and been removed. I can't see why this behaviour would be any different with the Askar backend.
> For #4, it appears that the process is failing when trying to update the credential exchange record because it doesn't exist. This could be because auto-respond was enabled and the exchange has already completed and been removed. I can't see why this behaviour would be any different with the Askar backend.

It's indeed different. I have two 0.7.3 agents, one running in indy the other in aksar, both have all auto-respond-credential-xxx flags set to false. The indy one works a s expected. The askar one receives the credential offer, but is not able to handle the credential request trigger because the record is missing. I double checked to see if maybe the exchange runs through automatically and the controller reacts on a wrong state, but thats not the case.
@etschelp Which Storage Backend are you running in the Acapy. Do you use Postgres or SQLite ?
> @etschelp Which Storage Backend are you running in the Acapy. Do you use Postgres or SQLite ?

Always Postgres
@etschelp  We saw also errors on the Postgres Backend regarding encoding problems with `utf-8` and askar. Did you tried it with sqlite ?
What are your encoding problems? So far I did not experience any. But your question made me curious, by looking at the init methods in the indy and askar sdk's I do not see that there is any encoding set. Which means for postgres it comes back to the defaults. If you are using the docker image its derived from `LANG=en_US.utf8 `if not it is ascii, or what was set during initdb, or during createdb. So the first thing that I would check is if the encoding of the wallet db that aca-py is using is correct. 
Askar (via sqlx) always sets the client encoding to utf8 when creating a connection. What kind of encoding problems are you seeing?
On my Mac I can run the performance demo with a test of 300 credential exchanges using the following (with Postgres.app installed): `POSTGRES=1 LEDGER_URL=http://test.bcovrin.vonx.io python3 -m runners.performance --wallet-type askar`. Note this uses the `issue-credential-2.0/send` endpoint to automate the send.

```sh
Startup duration: 5.05s
Faber      | Connected
Connect duration: 0.17s
Faber      | Publishing test schema
Alice      | Connected
Faber      | Schema ID: VYRv5TfbgWBuuzru4ayZkb:2:degree schema:42.9.95
Faber      | Publishing test credential definition
Faber      | Credential Definition ID: VYRv5TfbgWBuuzru4ayZkb:3:CL:131901:default
Publish duration: 5.73s
Faber      | Started 100 credential exchanges in 2.10s
Faber      | Started 100 credential exchanges in 3.75s
Faber      | Done starting 300 credential exchanges in 9.38s
Issuing credentials   100.0% [===================================================>] 300/300 eta [00:00] 
Receiving credentials 100.0% [===================================================>] 300/300 eta [00:00] 
Faber      | Completed 300 credential exchanges in 10.68s
Faber      | Average time per credential: 0.04s (28.09/s)
Alice      | alice602305 DB                 |             items |        items_tags
Alice      | ==========================================================================================
Alice      | 300 creds                      |      908   6568kB |     3914    848kB
Alice      | 
Faber      | faber446483 DB                 |             items |        items_tags
Faber      | ==========================================================================================
Faber      | 300 creds                      |       19     72kB |       45     72kB
```
@andrewwhitehead  @etschelp  We fixed by setting the `LANG` environment variable. I'll open a separate issue there we can discuss it not to derive from the topic. I think it's missing in the Docker Image.
Can we get an update on this one at the ACA-Pug meeting today/tomorrow (2022.02.08)?  We have a WIP PR and some back and forth that may address some of the issue.  Ideally, I'd like to get the issues remaining separated into separate issues, because who doesn't like having more issues? 😂 It is easier to track one issue per issue.

Resolving patch:
diff --git a/aries_cloudagent/revocation/routes.py b/aries_cloudagent/revocation/routes.py
--- a/aries_cloudagent/revocation/routes.py
+++ b/aries_cloudagent/revocation/routes.py
@@ -1252,7 +1252,7 @@ async def on_revocation_tails_file_event(profile: Profile, event: Event):
     revoc_reg_id = event.payload["context"]["rev_reg_id"]
     tails_local_path = tails_path(revoc_reg_id)
     (upload_success, reason) = await tails_server.upload_tails_file(
-        profile,
+        profile.context,
         revoc_reg_id,
         tails_local_path,
         interval=0.8,
diff --git a/aries_cloudagent/tails/indy_tails_server.py b/aries_cloudagent/tails/indy_tails_server.py
--- a/aries_cloudagent/tails/indy_tails_server.py
+++ b/aries_cloudagent/tails/indy_tails_server.py
@@ -2,6 +2,8 @@
 
 from typing import Tuple
 
+from ..config.injection_context import InjectionContext
+from ..ledger.multiple_ledger.base_manager import BaseMultipleLedgerManager
 from ..utils.http import put_file, PutError
 
 from .base import BaseTailsServer
@@ -13,7 +15,7 @@ class IndyTailsServer(BaseTailsServer):
 
     async def upload_tails_file(
         self,
-        context,
+        context: InjectionContext,
         rev_reg_id: str,
         tails_file_path: str,
         interval: float = 1.0,
@@ -30,9 +32,18 @@ async def upload_tails_file(
             backoff: exponential backoff in retry interval
             max_attempts: maximum number of attempts to make
         """
-
-        genesis_transactions = context.settings.get("ledger.genesis_transactions")
         tails_server_upload_url = context.settings.get("tails_server_upload_url")
+        genesis_transactions = context.settings.get("ledger.genesis_transactions")
+
+        if not genesis_transactions:
+            ledger_manager = context.injector.inject(BaseMultipleLedgerManager)
+            write_ledgers = await ledger_manager.get_write_ledger()
+            pool = write_ledgers[1].pool
+
+            try:
+                genesis_transactions = pool.genesis_transactions
+            except AttributeError:
+                genesis_transactions = pool.genesis_txns_cache
 
         if not tails_server_upload_url:
             raise TailsServerNotConfiguredError(


Test patch:
diff --git a/aries_cloudagent/tails/tests/test_indy.py b/aries_cloudagent/tails/tests/test_indy.py
--- a/aries_cloudagent/tails/tests/test_indy.py
+++ b/aries_cloudagent/tails/tests/test_indy.py
@@ -1,6 +1,8 @@
 from asynctest import mock as async_mock, TestCase as AsyncTestCase
 
 from ...config.injection_context import InjectionContext
+from ...core.in_memory import InMemoryProfile
+from ...ledger.multiple_ledger.base_manager import BaseMultipleLedgerManager
 
 from .. import indy_tails_server as test_module
 
@@ -38,6 +40,66 @@ async def test_upload(self):
             assert ok
             assert text == "tails-hash"
 
+    async def test_upload_indy_sdk(self):
+        profile = InMemoryProfile.test_profile()
+        profile.settings["tails_server_upload_url"] = "http://1.2.3.4:8088"
+        profile.context.injector.bind_instance(
+            BaseMultipleLedgerManager,
+            async_mock.MagicMock(
+                get_write_ledger=async_mock.CoroutineMock(
+                    return_value=(
+                        "test_ledger_id",
+                        async_mock.MagicMock(
+                            pool=async_mock.MagicMock(genesis_transactions="dummy")
+                        ),
+                    )
+                )
+            ),
+        )
+        indy_tails = test_module.IndyTailsServer()
+
+        with async_mock.patch.object(
+            test_module, "put_file", async_mock.CoroutineMock()
+        ) as mock_put:
+            mock_put.return_value = "tails-hash"
+            (ok, text) = await indy_tails.upload_tails_file(
+                profile.context,
+                REV_REG_ID,
+                "/tmp/dummy/path",
+            )
+            assert ok
+            assert text == "tails-hash"
+
+    async def test_upload_indy_vdr(self):
+        profile = InMemoryProfile.test_profile()
+        profile.settings["tails_server_upload_url"] = "http://1.2.3.4:8088"
+        profile.context.injector.bind_instance(
+            BaseMultipleLedgerManager,
+            async_mock.MagicMock(
+                get_write_ledger=async_mock.CoroutineMock(
+                    return_value=(
+                        "test_ledger_id",
+                        async_mock.MagicMock(
+                            pool=async_mock.MagicMock(genesis_txns_cache="dummy")
+                        ),
+                    )
+                )
+            ),
+        )
+        indy_tails = test_module.IndyTailsServer()
+
+        with async_mock.patch.object(
+            test_module, "put_file", async_mock.CoroutineMock()
+        ) as mock_put:
+            mock_put.return_value = "tails-hash"
+            (ok, text) = await indy_tails.upload_tails_file(
+                profile.context,
+                REV_REG_ID,
+                "/tmp/dummy/path",
+            )
+            assert ok
+            assert text == "tails-hash"
+
     async def test_upload_x(self):
         context = InjectionContext(
             settings={