Repository: deepjavalibrary/djl
Pull request #1877 (2022-08-08 03:04:18)

Adds more unit tests for huggingface translator

codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1877?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
> Merging [#1877](https://codecov.io/gh/deepjavalibrary/djl/pull/1877?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (dd9d0b0) into [master](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (bb5073f) will **decrease** coverage by `2.08%`.
> The diff coverage is `66.10%`.

```diff
@@             Coverage Diff              @@
##             master    #1877      +/-   ##
============================================
- Coverage     72.08%   69.99%   -2.09%     
- Complexity     5126     5578     +452     
============================================
  Files           473      529      +56     
  Lines         21970    24603    +2633     
  Branches       2351     2676     +325     
============================================
+ Hits          15838    17222    +1384     
- Misses         4925     6066    +1141     
- Partials       1207     1315     +108     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/1877?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [...odality/cv/translator/BigGANTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `33.33% <0.00%> (+8.33%)` | :arrow_up: |
| [...nslator/InstanceSegmentationTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW5zdGFuY2VTZWdtZW50YXRpb25UcmFuc2xhdG9yRmFjdG9yeS5qYXZh) | `14.28% <0.00%> (-3.90%)` | :arrow_down: |
| [.../cv/translator/SemanticSegmentationTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU2VtYW50aWNTZWdtZW50YXRpb25UcmFuc2xhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../cv/translator/StyleTransferTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU3R5bGVUcmFuc2ZlclRyYW5zbGF0b3JGYWN0b3J5LmphdmE=) | `40.00% <ø> (ø)` | |
| ... and [426 more](https://codecov.io/gh/deepjavalibrary/djl/pull/1877/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java
index 884460d8a..92586788c 100644
--- a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java
@@ -32,7 +32,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.concurrent.ConcurrentHashMap;
 
-/** The translator for Huggingface question answering model. */
+/** The translator for Huggingface fill mask model. */
 public class FillMaskTranslator implements Translator<String, Classifications> {
 
     private HuggingFaceTokenizer tokenizer;
@@ -100,7 +100,7 @@ public class FillMaskTranslator implements Translator<String, Classifications> {
     }
 
     /**
-     * Creates a builder to build a {@code QuestionAnsweringTranslator}.
+     * Creates a builder to build a {@code FillMaskTranslator}.
      *
      * @return a new builder
      */
@@ -109,7 +109,7 @@ public class FillMaskTranslator implements Translator<String, Classifications> {
     }
 
     /**
-     * Creates a builder to build a {@code QuestionAnsweringTranslator}.
+     * Creates a builder to build a {@code FillMaskTranslator}.
      *
      * @param arguments the models' arguments
      * @return a new builder
@@ -121,7 +121,7 @@ public class FillMaskTranslator implements Translator<String, Classifications> {
         return builder;
     }
 
-    /** The builder for question answering translator. */
+    /** The builder for fill mask translator. */
     public static final class Builder {
 
         private String maskedToken = "[MASK]";


Test patch:
diff --git a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
index fa85d05e8..c4f05d446 100644
--- a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
+++ b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
@@ -12,11 +12,14 @@
  */
 package ai.djl.huggingface.tokenizers;
 
+import ai.djl.Model;
 import ai.djl.ModelException;
 import ai.djl.huggingface.translator.FillMaskTranslatorFactory;
 import ai.djl.huggingface.translator.QuestionAnsweringTranslatorFactory;
 import ai.djl.inference.Predictor;
 import ai.djl.modality.Classifications;
+import ai.djl.modality.Input;
+import ai.djl.modality.Output;
 import ai.djl.modality.nlp.qa.QAInput;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
@@ -37,6 +40,8 @@ import java.io.IOException;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Map;
 
 public class TranslatorTest {
 
@@ -54,7 +59,6 @@ public class TranslatorTest {
                 "BBC Japan was a general entertainment Channel. "
                         + "Which operated between December 2004 and April 2006. "
                         + "It ceased operations after its Japanese distributor folded.";
-        QAInput input = new QAInput(question, paragraph);
 
         Block block =
                 new LambdaBlock(
@@ -63,9 +67,9 @@ public class TranslatorTest {
                             long[][] start = new long[1][36];
                             long[][] end = new long[1][36];
                             start[0][0] = 2;
-                            start[0][20] = 1;
+                            start[0][21] = 1;
                             end[0][0] = 2;
-                            end[0][21] = 1;
+                            end[0][20] = 1;
                             NDArray arr1 = manager.create(start);
                             NDArray arr2 = manager.create(end);
                             return new NDList(arr1, arr2);
@@ -73,6 +77,7 @@ public class TranslatorTest {
                         "model");
         Path modelDir = Paths.get("build/model");
         Files.createDirectories(modelDir);
+
         Criteria<QAInput, String> criteria =
                 Criteria.builder()
                         .setTypes(QAInput.class, String.class)
@@ -86,16 +91,57 @@ public class TranslatorTest {
 
         try (ZooModel<QAInput, String> model = criteria.loadModel();
                 Predictor<QAInput, String> predictor = model.newPredictor()) {
+            QAInput input = new QAInput(question, paragraph);
             String res = predictor.predict(input);
             Assert.assertEquals(res, "December 2004");
         }
+
+        Criteria<Input, Output> criteria2 =
+                Criteria.builder()
+                        .setTypes(Input.class, Output.class)
+                        .optModelPath(modelDir)
+                        .optBlock(block)
+                        .optEngine("PyTorch")
+                        .optArgument("tokenizer", "bert-base-cased")
+                        .optOption("hasParameter", "false")
+                        .optTranslatorFactory(new QuestionAnsweringTranslatorFactory())
+                        .build();
+
+        try (ZooModel<Input, Output> model = criteria2.loadModel();
+                Predictor<Input, Output> predictor = model.newPredictor()) {
+            Input input = new Input();
+            input.add("question", question);
+            input.add("paragraph", paragraph);
+            Output res = predictor.predict(input);
+            Assert.assertEquals(res.getAsString(0), "December 2004");
+        }
+
+        try (Model model = Model.newInstance("test")) {
+            model.setBlock(block);
+            Map<String, String> options = new HashMap<>();
+            options.put("hasParameter", "false");
+            model.load(modelDir, "test", options);
+
+            QuestionAnsweringTranslatorFactory factory = new QuestionAnsweringTranslatorFactory();
+            Map<String, String> arguments = new HashMap<>();
+
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+
+            arguments.put("tokenizer", "bert-base-cased");
+
+            Assert.assertThrows(
+                    IllegalArgumentException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+        }
     }
 
     @Test
     public void testFillMaskTranslator() throws ModelException, IOException, TranslateException {
         TestRequirements.notArm();
 
-        String input = "Hello I'm a [MASK] model.";
+        String text = "Hello I'm a [MASK] model.";
 
         Block block =
                 new LambdaBlock(
@@ -114,6 +160,7 @@ public class TranslatorTest {
                         "model");
         Path modelDir = Paths.get("build/model");
         Files.createDirectories(modelDir);
+
         Criteria<String, Classifications> criteria =
                 Criteria.builder()
                         .setTypes(String.class, Classifications.class)
@@ -127,8 +174,54 @@ public class TranslatorTest {
 
         try (ZooModel<String, Classifications> model = criteria.loadModel();
                 Predictor<String, Classifications> predictor = model.newPredictor()) {
-            Classifications res = predictor.predict(input);
+            Classifications res = predictor.predict(text);
             Assert.assertEquals(res.best().getClassName(), "fashion");
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> predictor.predict("Hello I'm a invalid model."));
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> predictor.predict("I'm a [MASK] [MASK] model."));
+        }
+
+        Criteria<Input, Output> criteria2 =
+                Criteria.builder()
+                        .setTypes(Input.class, Output.class)
+                        .optModelPath(modelDir)
+                        .optBlock(block)
+                        .optEngine("PyTorch")
+                        .optArgument("tokenizer", "bert-base-uncased")
+                        .optOption("hasParameter", "false")
+                        .optTranslatorFactory(new FillMaskTranslatorFactory())
+                        .build();
+
+        try (ZooModel<Input, Output> model = criteria2.loadModel();
+                Predictor<Input, Output> predictor = model.newPredictor()) {
+            Input input = new Input();
+            input.add(text);
+            Output out = predictor.predict(input);
+            Classifications res = (Classifications) out.getData();
+            Assert.assertEquals(res.best().getClassName(), "fashion");
+        }
+
+        try (Model model = Model.newInstance("test")) {
+            model.setBlock(block);
+            Map<String, String> options = new HashMap<>();
+            options.put("hasParameter", "false");
+            model.load(modelDir, "test", options);
+
+            FillMaskTranslatorFactory factory = new FillMaskTranslatorFactory();
+            Map<String, String> arguments = new HashMap<>();
+
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+
+            arguments.put("tokenizer", "bert-base-cased");
+
+            Assert.assertThrows(
+                    IllegalArgumentException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
         }
     }
 }