Repository: oasisprotocol/oasis-sdk
Pull request #51 (2021-03-20 00:07:02)

ts-web/rt tests with keyvalue runtime
uses #37, cc @abukosek 


pro-wh: can't get the testnet to work. pls help
pro-wh: this now runs locally. something's getting tied up in initialization or something, with the first `insert` call taking ~25 seconds.

might have some issue running on CI, because when I run it locally, it fails for a while with `client: not finished initial sync`. will need to add a wait for that
kostko: > this now runs locally. something's getting tied up in initialization or something, with the first insert call taking ~1 minute.

That's normal as it requires some initial epoch transitions to set up committees.
codecov[bot]: # [Codecov](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=h1) Report
> Merging [#51](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=desc) (0b48862) into [main](https://codecov.io/gh/oasisprotocol/oasis-sdk/commit/7f9d45d3c4ab09dae170a282ada6a693467a9492?el=desc) (7f9d45d) will **decrease** coverage by `3.82%`.
> The diff coverage is `0.00%`.

> :exclamation: Current head 0b48862 differs from pull request most recent head f0e601c. Consider uploading reports for the commit f0e601c to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51/graphs/tree.svg?width=650&height=150&src=pr&token=uscfoYjSZn)](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##             main      #51      +/-   ##
==========================================
- Coverage   56.14%   52.32%   -3.83%     
==========================================
  Files          24       26       +2     
  Lines        1122     1204      +82     
==========================================
  Hits          630      630              
- Misses        492      574      +82     
```


| [Impacted Files](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [tests/runtimes/simple-keyvalue/src/lib.rs](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51/diff?src=pr&el=tree#diff-dGVzdHMvcnVudGltZXMvc2ltcGxlLWtleXZhbHVlL3NyYy9saWIucnM=) | `0.00% <0.00%> (ø)` | |
| [tests/runtimes/simple-keyvalue/src/main.rs](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51/diff?src=pr&el=tree#diff-dGVzdHMvcnVudGltZXMvc2ltcGxlLWtleXZhbHVlL3NyYy9tYWluLnJz) | `0.00% <0.00%> (ø)` | |
| [runtime-sdk/src/runtime.rs](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51/diff?src=pr&el=tree#diff-cnVudGltZS1zZGsvc3JjL3J1bnRpbWUucnM=) | `0.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=footer). Last update [7f9d45d...f0e601c](https://codecov.io/gh/oasisprotocol/oasis-sdk/pull/51?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

pro-wh: getting stream reset from client-0's internal.sock on the first WaitReady call, according to envoy. but client-0's logs say it never even received a request. can't reproduce this locally.
pro-wh: seemed to be a CORS thing. the cypress test was pointed at the wrong port. dunno why it worked locally though
pro-wh: thanks, I'll rebase

Patch:
diff --git a/client-sdk/ts-web/rt/cypress/integration/playground.spec.js b/client-sdk/ts-web/rt/cypress/integration/playground.spec.js
index 75854570c9..9bb95d1496 100644
--- a/client-sdk/ts-web/rt/cypress/integration/playground.spec.js
+++ b/client-sdk/ts-web/rt/cypress/integration/playground.spec.js
@@ -4,7 +4,7 @@ import * as shared from '../../playground/src/shared';
 
 describe('playground', () => {
     it('finishes', () => {
-        cy.visit('http://localhost:8080/');
-        cy.contains(shared.CYPRESS_DONE_STRING, {timeout: 60_000});
+        cy.visit('http://localhost:42280/');
+        cy.contains(shared.CYPRESS_DONE_STRING, {timeout: 120_000});
     });
 });
diff --git a/client-sdk/ts-web/rt/playground/build-runtime.sh b/client-sdk/ts-web/rt/playground/build-runtime.sh
new file mode 100755
index 0000000000..ceba2f7f41
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/build-runtime.sh
@@ -0,0 +1,7 @@
+#!/bin/sh -eux
+if [ ! -e ../../../../target/debug/test-runtime-simple-keyvalue ]; then
+    (
+        cd ../../../..
+        cargo build -p test-runtime-simple-keyvalue
+    )
+fi
diff --git a/client-sdk/ts-web/rt/playground/consts.sh b/client-sdk/ts-web/rt/playground/consts.sh
new file mode 100644
index 0000000000..8e09415eee
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/consts.sh
@@ -0,0 +1,3 @@
+OASIS_CORE_VERSION=21.0.1
+BUILD_NUMBER=4371
+SIMPLE_KEYMANAGER_ARTIFACT=https://buildkite.com/organizations/oasisprotocol/pipelines/oasis-core-ci/builds/4371/jobs/f488e87f-abd1-442b-a20e-7b8dee75d06a/artifacts/812074d7-7e45-4e21-9975-377788948fa3
diff --git a/client-sdk/ts-web/rt/playground/download-artifacts.sh b/client-sdk/ts-web/rt/playground/download-artifacts.sh
new file mode 100755
index 0000000000..620330f4bb
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/download-artifacts.sh
@@ -0,0 +1,18 @@
+#!/bin/sh -eux
+. ./consts.sh
+
+mkdir -p untracked
+if [ ! -e "untracked/oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-node" ]; then
+    (
+        cd untracked
+        curl -fLO "https://github.com/oasisprotocol/oasis-core/releases/download/v$OASIS_CORE_VERSION/oasis_core_${OASIS_CORE_VERSION}_linux_amd64.tar.gz"
+        tar -xf "oasis_core_${OASIS_CORE_VERSION}_linux_amd64.tar.gz" \
+            "oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-node" \
+            "oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-net-runner" \
+            "oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-core-runtime-loader"
+    )
+fi
+if [ ! -e "untracked/simple-keymanager-$BUILD_NUMBER" ]; then
+    curl -fLo "untracked/simple-keymanager-$BUILD_NUMBER" "$SIMPLE_KEYMANAGER_ARTIFACT"
+    chmod +x "untracked/simple-keymanager-$BUILD_NUMBER"
+fi
diff --git a/client-sdk/ts-web/rt/playground/sample-envoy.yaml b/client-sdk/ts-web/rt/playground/sample-envoy.yaml
new file mode 100644
index 0000000000..97398864ed
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/sample-envoy.yaml
@@ -0,0 +1,62 @@
+static_resources:
+  listeners:
+  - name: listener_0
+    address:
+      socket_address:
+        address: 127.0.0.1
+        port_value: 42280
+    filter_chains:
+    - filters:
+      - name: envoy.filters.network.http_connection_manager
+        typed_config:
+          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
+          codec_type: AUTO
+          stat_prefix: ingress_http
+          route_config:
+            virtual_hosts:
+            - name: ts_dev_vh
+              domains: ['*']
+              routes:
+              - match:
+                  prefix: '/oasis-core'
+                route:
+                  cluster: oasis_node_grpc
+                  timeout: 0s
+                  max_stream_duration:
+                    grpc_timeout_header_max: 0s
+              - match:
+                  prefix: '/'
+                route:
+                  cluster: dev_static
+          http_filters:
+          - name: envoy.filters.http.grpc_web
+          - name: envoy.filters.http.router
+  clusters:
+  - name: oasis_node_grpc
+    connect_timeout: 0.25s
+    load_assignment:
+      cluster_name: cluster_0
+      endpoints:
+      - lb_endpoints:
+        - endpoint:
+            address:
+              pipe:
+                path: /tmp/oasis-net-runner-sdk-rt/net-runner/network/client-0/internal.sock
+    http2_protocol_options: {}
+  - name: dev_static
+    connect_timeout: 0.25s
+    load_assignment:
+      cluster_name: cluster_1
+      endpoints:
+      - lb_endpoints:
+        - endpoint:
+            address:
+              socket_address:
+                address: 127.0.0.1
+                port_value: 8080
+admin:
+  access_log_path: /dev/null
+  address:
+    socket_address:
+      address: 127.0.0.1
+      port_value: 42281
diff --git a/client-sdk/ts-web/rt/playground/sample-run-envoy.sh b/client-sdk/ts-web/rt/playground/sample-run-envoy.sh
new file mode 100755
index 0000000000..d5ab0d7af0
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/sample-run-envoy.sh
@@ -0,0 +1,2 @@
+#!/bin/sh -eux
+envoy -c sample-envoy.yaml
diff --git a/client-sdk/ts-web/rt/playground/sample-run-network.sh b/client-sdk/ts-web/rt/playground/sample-run-network.sh
new file mode 100755
index 0000000000..90aa014a09
--- /dev/null
+++ b/client-sdk/ts-web/rt/playground/sample-run-network.sh
@@ -0,0 +1,14 @@
+#!/bin/sh -eux
+. ./consts.sh
+
+./download-artifacts.sh
+./build-runtime.sh
+
+mkdir -p /tmp/oasis-net-runner-sdk-rt
+"./untracked/oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-net-runner" \
+    --fixture.default.node.binary "untracked/oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-node" \
+    --fixture.default.runtime.binary ../../../../target/debug/test-runtime-simple-keyvalue \
+    --fixture.default.runtime.loader "untracked/oasis_core_${OASIS_CORE_VERSION}_linux_amd64/oasis-core-runtime-loader" \
+    --fixture.default.keymanager.binary "untracked/simple-keymanager-$BUILD_NUMBER" \
+    --basedir /tmp/oasis-net-runner-sdk-rt \
+    --basedir.no_temp_dir
diff --git a/client-sdk/ts-web/rt/playground/src/index.js b/client-sdk/ts-web/rt/playground/src/index.js
index ac07e24f6b..6003529b92 100644
--- a/client-sdk/ts-web/rt/playground/src/index.js
+++ b/client-sdk/ts-web/rt/playground/src/index.js
@@ -1,8 +1,67 @@
 // @ts-check
 
+import * as oasis from '@oasisprotocol/client';
+
 import * as oasisRT from './../..';
 import * as shared from './shared';
 
+const KEYVALUE_RUNTIME_ID = oasis.misc.fromHex('8000000000000000000000000000000000000000000000000000000000000000');
+
+const FEE_FREE = /** @type {oasisRT.types.BaseUnits} */ ([oasis.quantity.fromBigInt(0n), oasisRT.token.NATIVE_DENOMINATION]);
+
+/**
+ * The name of our module.
+ */
+const MODULE_NAME = 'keyvalue';
+
+const ERR_INVALID_ARGUMENT_CODE = 1;
+
+// Callable methods.
+const METHOD_INSERT = 'keyvalue.Insert';
+const METHOD_REMOVE = 'keyvalue.Remove';
+// Queries.
+const METHOD_GET = 'keyvalue.Get';
+
+const EVENT_DUMMY_EVENT_CODE = 1;
+
+/**
+ * @typedef {object} Key
+ * @property {Uint8Array} key
+ */
+
+/**
+ * @typedef {object} KeyValue
+ * @property {Uint8Array} key
+ * @property {Uint8Array} value
+ */
+
+class Wrapper extends oasisRT.wrapper.Base {
+
+    /**
+     * @param {Uint8Array} runtimeID
+     */
+    constructor(runtimeID) {
+        super(runtimeID);
+    }
+
+    /**
+     * @returns {oasisRT.wrapper.TransactionWrapper<KeyValue, void>}
+     */
+    callInsert() { return this.call(METHOD_INSERT); }
+    /**
+     * @returns {oasisRT.wrapper.TransactionWrapper<Key, void>}
+     */
+    callRemove() { return this.call(METHOD_REMOVE); }
+    /**
+     * @returns {oasisRT.wrapper.QueryWrapper<Key, KeyValue>}
+     */
+    queryGet() { return this.query(METHOD_GET); }
+
+}
+
+const nic = new oasis.client.NodeInternal('http://localhost:42280');
+const keyvalueWrapper = new Wrapper(KEYVALUE_RUNTIME_ID);
+
 (async function () {
     try {
         // Try secp256k1 signing.
@@ -18,6 +77,63 @@ import * as shared from './shared';
             console.log('valid', await oasisRT.signatureSecp256k1.verify('test context', message, signature, publicKey));
         }
 
+        // Wait for ready.
+        {
+            console.log('waiting for node to be ready');
+            const waitStart = Date.now();
+            await nic.nodeControllerWaitReady();
+            const waitEnd = Date.now();
+            console.log(`ready ${waitEnd - waitStart} ms`);
+        }
+
+        // Try key-value runtime.
+        {
+            const THE_KEY = oasis.misc.fromString('greeting-js');
+            const THE_VALUE = oasis.misc.fromString('Hi from JavaScript');
+
+            const alice = oasis.signature.EllipticSigner.fromSecret(await oasis.hash.hash(oasis.misc.fromString('oasis-runtime-sdk/test-keys: alice')), 'this key is not important');
+            const csAlice = new oasis.signature.BlindContextSigner(alice);
+            // The keyvalue runtime does not use the accounts module, so there
+            // is no nonce checking.
+            const nonce = BigInt(Date.now());
+            const siAlice = /** @type {oasisRT.types.SignerInfo} */ ({pub: {ed25519: csAlice.public()}, nonce});
+
+            console.log('insert', THE_KEY, THE_VALUE);
+            const twInsert = keyvalueWrapper.callInsert()
+                .setBody({
+                    key: THE_KEY,
+                    value: THE_VALUE,
+                })
+                .setSignerInfo([siAlice])
+                .setFeeAmount(FEE_FREE)
+                .setFeeGas(0n);
+            await twInsert.sign([csAlice]);
+            await twInsert.submit(nic);
+            console.log('ok');
+
+            console.log('get', THE_KEY);
+            const getResult = await keyvalueWrapper.queryGet()
+                .setArgs({
+                    key: THE_KEY,
+                })
+                .query(nic);
+            console.log('ok', getResult.key, getResult.value);
+            if (oasis.misc.toHex(getResult.key) !== oasis.misc.toHex(THE_KEY)) throw new Error('Key mismatch');
+            if (oasis.misc.toHex(getResult.value) !== oasis.misc.toHex(THE_VALUE)) throw new Error('Value mismatch');
+
+            console.log('remove', THE_KEY);
+            const twRemove = keyvalueWrapper.callRemove()
+                .setBody({
+                    key: THE_KEY,
+                })
+                .setSignerInfo([siAlice])
+                .setFeeAmount(FEE_FREE)
+                .setFeeGas(0n);
+            await twRemove.sign([csAlice]);
+            await twRemove.submit(nic);
+            console.log('ok');
+        }
+
         // Tell cypress that we're done.
         {
             document.body.appendChild(document.createTextNode(shared.CYPRESS_DONE_STRING));


Test patch:
diff --git a/.github/workflows/ci-test.yaml b/.github/workflows/ci-test.yaml
index 46c0a5c45a..0cac534ac0 100644
--- a/.github/workflows/ci-test.yaml
+++ b/.github/workflows/ci-test.yaml
@@ -121,7 +121,7 @@ jobs:
       - name: 'proxy: Install Envoy'
         # https://www.getenvoy.io/install/envoy/ubuntu/
         run: |
-          curl -sL 'https://getenvoy.io/gpg' | sudo apt-key add -
+          curl -sL 'https://www.getenvoy.io/gpg' | sudo apt-key add -
           sudo add-apt-repository "deb [arch=amd64] https://dl.bintray.com/tetrate/getenvoy-deb $(lsb_release -cs) stable"
           sudo apt-get update && sudo apt-get install -y getenvoy-envoy
 
@@ -147,6 +147,13 @@ jobs:
         working-directory: client-sdk/ts-web/core
         run: npx -c 'cypress run'
 
+      - name: Upload screenshot
+        if: failure()
+        uses: actions/upload-artifact@v2
+        with:
+          name: e2e-ts-web-core-screenshots
+          path: client-sdk/ts-web/core/cypress/screenshots
+
   e2e-ts-web-rt:
     # NOTE: This name appears in GitHub's Checks API.
     name: e2e-ts-web-rt
@@ -179,6 +186,45 @@ jobs:
         working-directory: client-sdk/ts-web/rt
         run: npm run-script playground &
 
+      - name: 'proxy: Install Envoy'
+        # https://www.getenvoy.io/install/envoy/ubuntu/
+        run: |
+          curl -sL 'https://www.getenvoy.io/gpg' | sudo apt-key add -
+          sudo add-apt-repository "deb [arch=amd64] https://dl.bintray.com/tetrate/getenvoy-deb $(lsb_release -cs) stable"
+          sudo apt-get update && sudo apt-get install -y getenvoy-envoy
+
+      - name: 'proxy: Start'
+        working-directory: client-sdk/ts-web/rt/playground
+        run: ./sample-run-envoy.sh &
+
+      - name: 'testnet: Install system packages'
+        run: sudo apt-get -y install bubblewrap
+
+      - name: 'testnet: Download artifacts'
+        working-directory: client-sdk/ts-web/rt/playground
+        run: ./download-artifacts.sh
+
+      - name: 'testnet: Build runtime'
+        working-directory: client-sdk/ts-web/rt/playground
+        run: ./build-runtime.sh
+
+      - name: 'testnet: Start'
+        working-directory: client-sdk/ts-web/rt/playground
+        run: ./sample-run-network.sh &
+
+      - name: 'testnet: Wait for node socket'
+        run: |
+          while [ ! -e /tmp/oasis-net-runner-sdk-rt/net-runner/network/client-0/internal.sock ]; do
+            sleep 1
+          done
+
       - name: Cypress run
         working-directory: client-sdk/ts-web/rt
         run: npx -c 'cypress run'
+
+      - name: Upload screenshot
+        if: failure()
+        uses: actions/upload-artifact@v2
+        with:
+          name: e2e-ts-web-rt-screenshots
+          path: client-sdk/ts-web/rt/cypress/screenshots