Repository: deepjavalibrary/djl
Pull request #2148 (2022-11-09 21:25:03)

[api] Allows non-ServingTranslatorFactory for DJLServing
## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
Base: **72.08**% // Head: **71.53**% // Decreases project coverage by **`-0.55%`** :warning:
> Coverage data is based on head [(`27d1dfc`)](https://codecov.io/gh/deepjavalibrary/djl/pull/2148?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) compared to base [(`bb5073f`)](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> Patch coverage: 72.08% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##             master    #2148      +/-   ##
============================================
- Coverage     72.08%   71.53%   -0.56%     
- Complexity     5126     6324    +1198     
============================================
  Files           473      625     +152     
  Lines         21970    27913    +5943     
  Branches       2351     3010     +659     
============================================
+ Hits          15838    19967    +4129     
- Misses         4925     6479    +1554     
- Partials       1207     1467     +260     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/2148?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [.../modality/cv/translator/ImageFeatureExtractor.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW1hZ2VGZWF0dXJlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../ai/djl/modality/cv/translator/YoloTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3IuamF2YQ==) | `27.77% <0.00%> (+18.95%)` | :arrow_up: |
| [...modality/cv/translator/wrapper/FileTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3Ivd3JhcHBlci9GaWxlVHJhbnNsYXRvci5qYXZh) | `44.44% <ø> (ø)` | |
| [...y/cv/translator/wrapper/InputStreamTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3Ivd3JhcHBlci9JbnB1dFN0cmVhbVRyYW5zbGF0b3IuamF2YQ==) | `44.44% <ø> (ø)` | |
| ... and [559 more](https://codecov.io/gh/deepjavalibrary/djl/pull/2148/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2148?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/api/src/main/java/ai/djl/ndarray/internal/NDFormat.java b/api/src/main/java/ai/djl/ndarray/internal/NDFormat.java
index 2bb36da49..51e49f6ab 100644
--- a/api/src/main/java/ai/djl/ndarray/internal/NDFormat.java
+++ b/api/src/main/java/ai/djl/ndarray/internal/NDFormat.java
@@ -123,6 +123,7 @@ public abstract class NDFormat {
             sb.append("[]").append(LF);
         } else if (dimension == 0) {
             // scalar case
+            init(array);
             sb.append(format(array.toArray()[0])).append(LF);
         } else if (size > maxSize) {
             sb.append("[ Exceed max print size ]");
diff --git a/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java b/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
index dfaa514f8..ae728a981 100644
--- a/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
+++ b/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
@@ -72,9 +72,7 @@ public class ServingTranslatorFactory implements TranslatorFactory {
         String factoryClass = ArgumentsUtil.stringValue(arguments, "translatorFactory");
         if (factoryClass != null && !factoryClass.isEmpty()) {
             TranslatorFactory factory = loadTranslatorFactory(factoryClass);
-            if (factory != null
-                    && !(factory instanceof ServingTranslatorFactory)
-                    && factory.isSupported(input, output)) {
+            if (factory != null && factory.isSupported(input, output)) {
                 logger.info("Using TranslatorFactory: {}", factory.getClass().getName());
                 return factory.newInstance(input, output, model, arguments);
             }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtModel.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtModel.java
index 49f1a1f6a..31088e1e2 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtModel.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtModel.java
@@ -78,14 +78,14 @@ public class PtModel extends BaseModel {
             String[] extraFileKeys = new String[0];
             String[] extraFileValues = new String[0];
             boolean mapLocation = false;
-            boolean retrain = false;
+            boolean trainParam = false;
             // load jit extra files
             if (options != null) {
                 if (options.containsKey("extraFiles")) {
                     extraFileKeys = ((String) options.get("extraFiles")).split(",");
                     extraFileValues = new String[extraFileKeys.length];
                 }
-                retrain = Boolean.parseBoolean((String) options.get("retrain"));
+                trainParam = Boolean.parseBoolean((String) options.get("trainParam"));
                 mapLocation = Boolean.parseBoolean((String) options.get("mapLocation"));
             }
             block =
@@ -95,15 +95,16 @@ public class PtModel extends BaseModel {
                             mapLocation,
                             extraFileKeys,
                             extraFileValues,
-                            retrain);
+                            trainParam);
             for (int i = 0; i < extraFileKeys.length; i++) {
                 properties.put(extraFileKeys[i], extraFileValues[i]);
             }
-            // By default, the parameters are frozen, since before adding this training feature
-            // `retrain`, it was frozen already by setting `JITCallGuard guard`, which disables
+            // By default, the parameters are frozen, since the previous version before adding this
+            // `trainParam`, they were frozen due to the setting `JITCallGuard guard`, which
+            // disables
             // autograd. Also, the pretrained parameters usually should not be updated too much. It
             // is safe to freeze it. Users may unfreeze it and set their learning rate small.
-            block.freezeParameters(!retrain);
+            block.freezeParameters(!trainParam);
         } else {
             boolean hasParameter = true;
             if (options != null) {
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
index eae7b68af..725c912b3 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
@@ -1555,7 +1555,7 @@ public final class JniUtils {
             boolean mapLocation,
             String[] extraFileKeys,
             String[] extraFileValues,
-            boolean retrain) {
+            boolean trainParam) {
         Device device = manager.getDevice();
         // MPS doesn't support mapLocation
         if ("mps".equals(device.getDeviceType())) {
@@ -1570,7 +1570,7 @@ public final class JniUtils {
                         mapLocation,
                         extraFileKeys,
                         extraFileValues,
-                        retrain);
+                        trainParam);
         return new PtSymbolBlock(manager, handle);
     }
 
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
index 7bab3e154..fc1f3f315 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
@@ -494,7 +494,7 @@ final class PyTorchLibrary {
             boolean mapLocation,
             String[] extraFileNames,
             String[] extraFileValues,
-            boolean retrain);
+            boolean trainParam);
 
     native long moduleLoad(
             InputStream is, int[] device, boolean mapLocation, byte[] buffer, long size);
diff --git a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_inference.cc b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_inference.cc
index 11c520a92..1c6911d20 100644
--- a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_inference.cc
+++ b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_inference.cc
@@ -32,7 +32,7 @@ struct JITCallGuard {
 JNIEXPORT jlong JNICALL
 Java_ai_djl_pytorch_jni_PyTorchLibrary_moduleLoad__Ljava_lang_String_2_3IZ_3Ljava_lang_String_2_3Ljava_lang_String_2Z(
     JNIEnv* env, jobject jthis, jstring jpath, jintArray jarray, jboolean jmap_location, jobjectArray jefnames,
-    jobjectArray jefvalues, jboolean jretrain) {
+    jobjectArray jefvalues, jboolean jtrainParam) {
   API_BEGIN()
   const std::string path = djl::utils::jni::GetStringFromJString(env, jpath);
   const torch::Device device = utils::GetDeviceFromJDevice(env, jarray);
@@ -44,7 +44,7 @@ Java_ai_djl_pytorch_jni_PyTorchLibrary_moduleLoad__Ljava_lang_String_2_3IZ_3Ljav
     map[name] = "";
   }
 
-  if (!jretrain) {
+  if (!jtrainParam) {
     JITCallGuard guard;
     torch::jit::Module module;
     if (jmap_location) {
diff --git a/examples/src/main/java/ai/djl/examples/training/transferlearning/TransferFreshFruit.java b/examples/src/main/java/ai/djl/examples/training/transferlearning/TransferFreshFruit.java
index 2f2b85c8e..cd0bfb792 100644
--- a/examples/src/main/java/ai/djl/examples/training/transferlearning/TransferFreshFruit.java
+++ b/examples/src/main/java/ai/djl/examples/training/transferlearning/TransferFreshFruit.java
@@ -66,13 +66,16 @@ public final class TransferFreshFruit {
     public static TrainingResult runExample(String[] args)
             throws IOException, TranslateException, ModelException, URISyntaxException {
         boolean retrain = args.length == 1 && "-p".equals(args[0]);
+        // The modelUrl can be replaced by local model path:
+        // String modelUrl = "/YOUR_PATH/resnet18_embedding.pt";
+        String modelUrl = "djl://ai.djl.pytorch/resnet18_embedding";
         Criteria<NDList, NDList> criteria =
                 Criteria.builder()
                         .setTypes(NDList.class, NDList.class)
-                        .optModelUrls("djl://ai.djl.pytorch/resnet18_embedding")
+                        .optModelUrls(modelUrl)
                         .optEngine("PyTorch")
                         .optProgress(new ProgressBar())
-                        .optOption("retrain", String.valueOf(retrain))
+                        .optOption("trainParam", String.valueOf(retrain))
                         .build();
 
         ZooModel<NDList, NDList> embedding = criteria.loadModel();
@@ -132,8 +135,8 @@ public final class TransferFreshFruit {
         float[] std = {0.229f, 0.224f, 0.225f};
 
         // To use local dataset, users can load it as follows
-        // Repository repository = Repository.newInstance("banana",
-        // Paths.get("local_data_root/banana/train"));
+        // `Repository repository = Repository.newInstance("banana",
+        // Paths.get("local_data_root/banana/train"));`
         // Then add the setting `.optRepository(repository)` to the builder below
         FruitsFreshAndRotten dataset =
                 FruitsFreshAndRotten.builder()
diff --git a/jupyter/tutorial/02_train_your_first_model.ipynb b/jupyter/tutorial/02_train_your_first_model.ipynb
index 875534e23..acac089c3 100644
--- a/jupyter/tutorial/02_train_your_first_model.ipynb
+++ b/jupyter/tutorial/02_train_your_first_model.ipynb
@@ -20,13 +20,13 @@
    "outputs": [],
    "source": [
     "// Add the snapshot repository to get the DJL snapshot artifacts\n",
-    "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n",
+    "%mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n",
     "\n",
     "// Add the maven dependencies\n",
-    "%maven ai.djl:api:0.19.0\n",
-    "%maven ai.djl:basicdataset:0.19.0\n",
-    "%maven ai.djl:model-zoo:0.19.0\n",
-    "%maven ai.djl.mxnet:mxnet-engine:0.19.0\n",
+    "%maven ai.djl:api:0.20.0-SNAPSHOT\n",
+    "%maven ai.djl:basicdataset:0.20.0-SNAPSHOT\n",
+    "%maven ai.djl:model-zoo:0.20.0-SNAPSHOT\n",
+    "%maven ai.djl.mxnet:mxnet-engine:0.20.0-SNAPSHOT\n",
     "%maven org.slf4j:slf4j-simple:1.7.32"
    ]
   },


Test patch:
diff --git a/api/src/test/java/ai/djl/ndarray/internal/NDFormatTest.java b/api/src/test/java/ai/djl/ndarray/internal/NDFormatTest.java
index 9b3f0294c..970ab0186 100644
--- a/api/src/test/java/ai/djl/ndarray/internal/NDFormatTest.java
+++ b/api/src/test/java/ai/djl/ndarray/internal/NDFormatTest.java
@@ -144,6 +144,9 @@ public class NDFormatTest {
             String str = large.toString();
             Assert.assertEquals(
                     str, "ND: (100, 100, 100, 100) cpu() float32\n[ Exceed max print size ]");
+
+            NDArray arr = manager.create(new float[] {1, 0, 0, 0});
+            Assert.assertEquals(arr.mean().toString(), "ND: () cpu() float32\n0.25\n");
         }
     }
 }
diff --git a/integration/src/test/java/ai/djl/integration/IntegrationTests.java b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
index 93cd6c12d..3ee337282 100644
--- a/integration/src/test/java/ai/djl/integration/IntegrationTests.java
+++ b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
@@ -46,7 +46,6 @@ public class IntegrationTests {
         for (String engine : engines) {
             System.setProperty("ai.djl.default_engine", engine);
             logger.info("Testing engine: {} ...", engine);
-            logger.info("=============== {} ===============", engine);
             Assert.assertTrue(new IntegrationTest(IntegrationTest.class).runTests(args));
             // currently each engine will reserve a certain amount of memory and hold it until
             // process terminate so running 3 different engines sequentially without