Repository: deepjavalibrary/djl
Pull request #1430 (2021-12-23 19:29:12)

[pytorch] Fixes unit test failure on GPU.
Change-Id: Ia2d1271ad97ccc2f59059981a7534e2470160943

## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
> Merging [#1430](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (ad6efd3) into [master](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (bb5073f) will **decrease** coverage by `1.44%`.
> The diff coverage is `47.78%`.

[![Impacted file tree graph](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/graphs/tree.svg?width=650&height=150&src=pr&token=mDKVNwFXRV&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

```diff
@@             Coverage Diff              @@
##             master    #1430      +/-   ##
============================================
- Coverage     72.08%   70.64%   -1.45%     
- Complexity     5126     5291     +165     
============================================
  Files           473      493      +20     
  Lines         21970    23208    +1238     
  Branches       2351     2530     +179     
============================================
+ Hits          15838    16395     +557     
- Misses         4925     5537     +612     
- Partials       1207     1276      +69     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <ø> (-5.24%)` | :arrow_down: |
| [...odality/cv/translator/BigGANTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `33.33% <0.00%> (+8.33%)` | :arrow_up: |
| [...nslator/InstanceSegmentationTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW5zdGFuY2VTZWdtZW50YXRpb25UcmFuc2xhdG9yRmFjdG9yeS5qYXZh) | `14.28% <0.00%> (-3.90%)` | :arrow_down: |
| [.../modality/cv/translator/YoloTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3JGYWN0b3J5LmphdmE=) | `8.33% <0.00%> (-1.67%)` | :arrow_down: |
| [...i/djl/modality/cv/translator/YoloV5Translator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1Y1VHJhbnNsYXRvci5qYXZh) | `5.69% <0.00%> (ø)` | |
| [...odality/cv/translator/YoloV5TranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1Y1VHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `8.33% <0.00%> (-1.67%)` | :arrow_down: |
| [...modality/nlp/embedding/TrainableWordEmbedding.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L25scC9lbWJlZGRpbmcvVHJhaW5hYmxlV29yZEVtYmVkZGluZy5qYXZh) | `45.00% <0.00%> (-3.34%)` | :arrow_down: |
| [...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL25kYXJyYXkvQnl0ZXNTdXBwbGllci5qYXZh) | `54.54% <0.00%> (-12.13%)` | :arrow_down: |
| [...i/src/main/java/ai/djl/ndarray/NDArrayAdapter.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL25kYXJyYXkvTkRBcnJheUFkYXB0ZXIuamF2YQ==) | `15.23% <0.00%> (-0.83%)` | :arrow_down: |
| ... and [156 more](https://codecov.io/gh/deepjavalibrary/djl/pull/1430/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Last update [6c7acba...ad6efd3](https://codecov.io/gh/deepjavalibrary/djl/pull/1430?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValue.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValue.java
index 472c6bdde..45b5866d9 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValue.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValue.java
@@ -250,6 +250,9 @@ public class IValue extends NativeResource<Long> {
      * @return a new {@code IValue} of type {@code NDArray[]}
      */
     public static IValue listFrom(IValue... list) {
+        if (list.length == 0) {
+            throw new IllegalArgumentException("Empty IValue list is not supported.");
+        }
         long[] tensors = Arrays.stream(list).mapToLong(IValue::getHandle).toArray();
         return new IValue(PyTorchLibrary.LIB.iValueFromList(tensors));
     }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValueUtils.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValueUtils.java
index 41cb8f57c..91f728b9e 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValueUtils.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/IValueUtils.java
@@ -73,7 +73,7 @@ public final class IValueUtils {
                 });
     }
 
-    private static IValue[] getInputs(NDList ndList) {
+    static IValue[] getInputs(NDList ndList) {
         List<PairList<String, PtNDArray>> outputs = new ArrayList<>();
         Map<String, Integer> indexMap = new ConcurrentHashMap<>();
         for (NDArray array : ndList) {
@@ -87,6 +87,10 @@ public final class IValueUtils {
                 int index = addToMap(indexMap, name, outputs);
                 PairList<String, PtNDArray> pl = outputs.get(index);
                 pl.add("[]", (PtNDArray) array);
+            } else if (name != null && Pattern.matches("\\w+\\(\\)", name)) {
+                int index = addToMap(indexMap, name, outputs);
+                PairList<String, PtNDArray> pl = outputs.get(index);
+                pl.add("()", (PtNDArray) array);
             } else {
                 PairList<String, PtNDArray> pl = new PairList<>();
                 pl.add(null, (PtNDArray) array);
@@ -98,12 +102,16 @@ public final class IValueUtils {
             PairList<String, PtNDArray> pl = outputs.get(i);
             String key = pl.get(0).getKey();
             if (key == null) {
-                // not List, Dict input
+                // not List, Dict, Tuple input
                 ret[i] = IValue.from(pl.get(0).getValue());
             } else if ("[]".equals(key)) {
                 // list
                 PtNDArray[] arrays = pl.values().toArray(new PtNDArray[0]);
                 ret[i] = IValue.listFrom(arrays);
+            } else if ("()".equals(key)) {
+                // Tuple
+                IValue[] arrays = pl.values().stream().map(IValue::from).toArray(IValue[]::new);
+                ret[i] = IValue.tupleFrom(arrays);
             } else {
                 Map<String, PtNDArray> map = new ConcurrentHashMap<>();
                 for (Pair<String, PtNDArray> pair : pl) {
diff --git a/website/demo.html b/website/demo.html
index 0110f624e..2eb4b3d6f 100644
--- a/website/demo.html
+++ b/website/demo.html
@@ -150,7 +150,7 @@
                             <li>
                                 <div class="collapsible-header"><i class="material-icons">filter_drama</i>Run Inference with DJL</div>
                                 <div class="collapsible-body">
-                                    <script src="https://gist.github.com/lanking520/2eaaf3bba094b10aaffebd999bd4ae88.js"></script>
+                                    <script src="https://gist.github.com/frankfliu/550f284c0e990e123ba95915219a56ac.js"></script>
                                     <p>You can copy with the code snippet above to run inference with MXNet. You can also try to change the Model URL to run inference on PyTorch and TensorFlow:</p>
                                     <p>PyTorch: Please select <b>PyTorch</b> in the engine selection dropdown and uncomment the <b>second modelUrl</b>. </p>
                                     <p>TensorFlow: Please select <b>TensorFlow</b> in the engine selection dropdown and uncomment the <b>third modelUrl</b>. </p>
@@ -236,7 +236,7 @@
 <script src="https://djl.ai/website/js/init.js"></script>
 <script src="https://cdn.jsdelivr.net/npm/vue"></script>
 <script src="./js/index.js"></script>
-    
+
 <script src="https://cdnjs.cloudflare.com/ajax/libs/xterm/3.14.5/xterm.min.js"></script>
 <script src="https://djl.ai/website/js/terminal.js"></script>
 <script src="https://djl.ai/website/js/doodle_draw.js"></script>


Test patch:
diff --git a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/IValueTest.java b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/IValueTest.java
index bb4f78b38..eebf7a159 100644
--- a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/IValueTest.java
+++ b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/IValueTest.java
@@ -137,6 +137,7 @@ public class IValueTest {
                 NDList list = ivalue.toNDList(manager);
                 Assert.assertEquals(list.size(), 3);
             }
+            Assert.assertThrows(() -> IValue.listFrom(new IValue[0]));
 
             try (IValue iv1 = IValue.from(array1);
                     IValue iv2 = IValue.from(array2);
diff --git a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/PtModelTest.java b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/PtModelTest.java
index ab2bcf16b..880a6fd81 100644
--- a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/PtModelTest.java
+++ b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/integration/PtModelTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.pytorch.integration;
 
+import ai.djl.Device;
 import ai.djl.Model;
 import ai.djl.ModelException;
 import ai.djl.inference.Predictor;
@@ -42,7 +43,8 @@ public class PtModelTest {
                         .build();
         try (ZooModel<NDList, NDList> zooModel = criteria.loadModel()) {
             Path modelFile = zooModel.getModelPath().resolve("traced_resnet18.pt");
-            try (PtModel model = (PtModel) Model.newInstance("test model")) {
+            // This model only support CPU
+            try (PtModel model = (PtModel) Model.newInstance("test model", Device.cpu())) {
                 model.load(Files.newInputStream(modelFile));
                 try (Predictor<NDList, NDList> predictor =
                         model.newPredictor(new NoopTranslator())) {
diff --git a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/IValueUtilsTest.java b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/IValueUtilsTest.java
new file mode 100644
index 000000000..85b2bc94a
--- /dev/null
+++ b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/IValueUtilsTest.java
@@ -0,0 +1,43 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.pytorch.jni;
+
+import ai.djl.ndarray.NDList;
+import ai.djl.ndarray.NDManager;
+import ai.djl.ndarray.types.Shape;
+import ai.djl.pytorch.engine.PtNDArray;
+import ai.djl.pytorch.engine.PtNDManager;
+import java.util.Arrays;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+public class IValueUtilsTest {
+    @Test
+    public void getInputsTestTupleSyntax() {
+        try (PtNDManager manager = (PtNDManager) NDManager.newBaseManager()) {
+            PtNDArray array1 = (PtNDArray) manager.zeros(new Shape(1));
+            array1.setName("Test()");
+            PtNDArray array2 = (PtNDArray) manager.ones(new Shape(1));
+            array2.setName("Test()");
+            NDList input = new NDList(array1, array2);
+            input.attach(manager);
+
+            IValue[] iValues = IValueUtils.getInputs(input);
+            Assert.assertEquals(iValues.length, 1);
+            Assert.assertTrue(iValues[0].isTuple());
+            Assert.assertEquals(iValues[0].toIValueTuple().length, 2);
+
+            Arrays.stream(iValues).forEach(IValue::close);
+        }
+    }
+}
diff --git a/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/package-info.java b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/package-info.java
new file mode 100644
index 000000000..cbf682843
--- /dev/null
+++ b/engines/pytorch/pytorch-engine/src/test/java/ai/djl/pytorch/jni/package-info.java
@@ -0,0 +1,14 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+/** The integration test for testing PyTorch specific features. */
+package ai.djl.pytorch.jni;