Repository: deepjavalibrary/djl
Pull request #1868 (2022-08-04 07:36:02)

[api] Adds DeferredTranslatorFactory to let serving.properties take effect
Change-Id: I656ec0a0f67176bb0de85763c04e6690db7148a6

## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1868?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
> Merging [#1868](https://codecov.io/gh/deepjavalibrary/djl/pull/1868?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (d5a1df7) into [master](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (bb5073f) will **decrease** coverage by `2.32%`.
> The diff coverage is `65.14%`.

```diff
@@             Coverage Diff              @@
##             master    #1868      +/-   ##
============================================
- Coverage     72.08%   69.76%   -2.33%     
- Complexity     5126     5553     +427     
============================================
  Files           473      528      +55     
  Lines         21970    24495    +2525     
  Branches       2351     2670     +319     
============================================
+ Hits          15838    17088    +1250     
- Misses         4925     6097    +1172     
- Partials       1207     1310     +103     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/1868?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [...odality/cv/translator/BigGANTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `33.33% <0.00%> (+8.33%)` | :arrow_up: |
| [...nslator/InstanceSegmentationTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW5zdGFuY2VTZWdtZW50YXRpb25UcmFuc2xhdG9yRmFjdG9yeS5qYXZh) | `14.28% <0.00%> (-3.90%)` | :arrow_down: |
| [.../cv/translator/SemanticSegmentationTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU2VtYW50aWNTZWdtZW50YXRpb25UcmFuc2xhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../cv/translator/StyleTransferTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU3R5bGVUcmFuc2ZlclRyYW5zbGF0b3JGYWN0b3J5LmphdmE=) | `40.00% <ø> (ø)` | |
| ... and [424 more](https://codecov.io/gh/deepjavalibrary/djl/pull/1868/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)


Patch:
diff --git a/.github/workflows/native_s3_huggingface.yml b/.github/workflows/native_s3_huggingface.yml
index c57a96637..16fc9bfee 100644
--- a/.github/workflows/native_s3_huggingface.yml
+++ b/.github/workflows/native_s3_huggingface.yml
@@ -39,13 +39,15 @@ jobs:
 
   build-tokenizers-jni-linux:
     runs-on: ubuntu-latest
-    container: amazonlinux:2
+    container: centos:centos7
     steps:
       - name: Install Environment
         run: |
           yum -y update
-          yum -y groupinstall "Development Tools"
-          yum -y install python3-pip perl-IPC-Cmd
+          yum -y install centos-release-scl-rh epel-release
+          yum -y install devtoolset-7 git patch cmake3 libstdc++-static
+          ln -s /usr/bin/cmake3 /usr/bin/cmake
+          curl https://sh.rustup.rs -sSf | sh -s -- -y
           pip3 install awscli --upgrade
       - uses: actions-rs/toolchain@v1
         with:
@@ -63,6 +65,8 @@ jobs:
             ${{ runner.os }}-gradle-
       - name: Release JNI prep
         run: |
+          source "$HOME/.cargo/env"
+          export PATH=$PATH:/opt/rh/devtoolset-7/root/usr/bin
           ./gradlew :extensions:tokenizers:compileJNI
           PYTORCH_PRECXX11=true ./gradlew -Pjni :extensions:tokenizers:test
       - name: Configure AWS Credentials
diff --git a/android/README.md b/android/README.md
index 9da89e238..0786d5ce7 100644
--- a/android/README.md
+++ b/android/README.md
@@ -5,6 +5,7 @@ It has the following two modules:
 
 - Core package: contains some Image processing toolkit for Android user using DJL
 - PyTorch Native: contains DJL PyTorch Android native package
+- ONNXRuntime: contains DJL ONNXRuntime Android package
 
 ## Installation
 You need to have Android SDK and Android NDK installed on your machine.
@@ -15,9 +16,10 @@ In gradle, you can include the snapshot repository and add the 4 modules in your
 
 ```
 dependencies {
-    implementation "ai.djl:api:0.18.0-SNAPSHOT"
-    implementation "ai.djl.android:core:0.18.0-SNAPSHOT"
-    androidRuntimeOnly "ai.djl.pytorch:pytorch-engine:0.18.0-SNAPSHOT"
-    androidRuntimeOnly "ai.djl.android:pytorch-native:0.18.0-SNAPSHOT"
+    implementation "ai.djl:api:0.19.0-SNAPSHOT"
+    implementation "ai.djl.android:core:0.19.0-SNAPSHOT"
+    androidRuntimeOnly "ai.djl.pytorch:pytorch-engine:0.19.0-SNAPSHOT"
+    androidRuntimeOnly "ai.djl.android:pytorch-native:0.19.0-SNAPSHOT"
+    androidRuntimeOnly "ai.djl.android:onnxruntime:0.19.0-SNAPSHOT"
 }
 ```
diff --git a/android/build.gradle b/android/build.gradle
index d469c4627..cc13cf3c0 100644
--- a/android/build.gradle
+++ b/android/build.gradle
@@ -23,6 +23,8 @@ allprojects {
             url "https://oss.sonatype.org/content/repositories/snapshots/"
         }
     }
+
+    apply from: file("${rootProject.projectDir}/../tools/gradle/android-formatter.gradle")
 }
 
 task clean(type: Delete) {
diff --git a/android/core/build.gradle b/android/core/build.gradle
index 4ab1e3783..0a31876cf 100644
--- a/android/core/build.gradle
+++ b/android/core/build.gradle
@@ -102,6 +102,7 @@ dependencies {
     testImplementation 'junit:junit:4.13.2'
     androidTestRuntimeOnly "ai.djl.pytorch:pytorch-engine:${version}"
     androidTestRuntimeOnly "ai.djl.android:pytorch-native:${version}"
+    androidTestImplementation "ai.djl.android:onnxruntime:${version}"
     androidTestImplementation 'androidx.test.ext:junit:1.1.3'
     androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
 }
diff --git a/android/core/src/main/java/ai/djl/android/core/BitmapImageFactory.java b/android/core/src/main/java/ai/djl/android/core/BitmapImageFactory.java
index aeca1d997..87e2f6485 100644
--- a/android/core/src/main/java/ai/djl/android/core/BitmapImageFactory.java
+++ b/android/core/src/main/java/ai/djl/android/core/BitmapImageFactory.java
@@ -19,14 +19,6 @@ import android.graphics.Color;
 import android.graphics.Paint;
 import android.graphics.Rect;
 
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.net.URL;
-import java.nio.ByteBuffer;
-import java.nio.file.Path;
-import java.util.List;
-
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.ImageFactory;
 import ai.djl.modality.cv.output.BoundingBox;
@@ -41,9 +33,15 @@ import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.util.RandomUtils;
 
-/**
- * {@code BitmapImageFactory} is the Android implementation of {@link ImageFactory}.
- */
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URL;
+import java.nio.ByteBuffer;
+import java.nio.file.Path;
+import java.util.List;
+
+/** {@code BitmapImageFactory} is the Android implementation of {@link ImageFactory}. */
 public class BitmapImageFactory extends ImageFactory {
 
     /** {@inheritDoc} */
@@ -59,7 +57,7 @@ public class BitmapImageFactory extends ImageFactory {
     /** {@inheritDoc} */
     @Override
     public Image fromUrl(URL url) throws IOException {
-            return fromInputStream(url.openStream());
+        return fromInputStream(url.openStream());
     }
 
     /** {@inheritDoc} */
@@ -122,7 +120,6 @@ public class BitmapImageFactory extends ImageFactory {
         return new BitMapWrapper(bitmap);
     }
 
-
     static class BitMapWrapper implements Image {
         private Bitmap bitmap;
 
@@ -195,7 +192,7 @@ public class BitmapImageFactory extends ImageFactory {
         @Override
         public void save(OutputStream os, String type) throws IOException {
             if (!bitmap.compress(Bitmap.CompressFormat.valueOf(type.toUpperCase()), 100, os)) {
-                throw new IOException("Cannot save image file to output stream File type " +  type);
+                throw new IOException("Cannot save image file to output stream File type " + type);
             }
         }
 
@@ -210,7 +207,7 @@ public class BitmapImageFactory extends ImageFactory {
         @Override
         public void drawBoundingBoxes(DetectedObjects detections) {
             Bitmap mutableBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
-            Canvas canvas =  new Canvas(mutableBitmap);
+            Canvas canvas = new Canvas(mutableBitmap);
             // set the paint configure
             int stroke = 2;
             Paint paint = new Paint();
@@ -235,7 +232,8 @@ public class BitmapImageFactory extends ImageFactory {
                         x,
                         y,
                         x + (int) (rectangle.getWidth() * imageWidth),
-                        y + (int) (rectangle.getHeight() * imageHeight), paint);
+                        y + (int) (rectangle.getHeight() * imageHeight),
+                        paint);
                 drawText(canvas, color, className, x, y, stroke, 4);
                 // If we have a mask instead of a plain rectangle, draw tha mask
                 if (box instanceof Mask) {
@@ -252,7 +250,7 @@ public class BitmapImageFactory extends ImageFactory {
         @Override
         public void drawJoints(Joints joints) {
             Bitmap mutableBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
-            Canvas canvas =  new Canvas(mutableBitmap);
+            Canvas canvas = new Canvas(mutableBitmap);
             // set the paint configure
             Paint paint = new Paint();
             paint.setStrokeWidth(2);
@@ -274,7 +272,8 @@ public class BitmapImageFactory extends ImageFactory {
         }
 
         private int randomColor() {
-            return Color.rgb(RandomUtils.nextInt(255), RandomUtils.nextInt(255), RandomUtils.nextInt(255));
+            return Color.rgb(
+                    RandomUtils.nextInt(255), RandomUtils.nextInt(255), RandomUtils.nextInt(255));
         }
 
         private int darker(int color) {
@@ -282,13 +281,11 @@ public class BitmapImageFactory extends ImageFactory {
             int r = Math.round(Color.red(color) * 0.8f);
             int g = Math.round(Color.green(color) * 0.8f);
             int b = Math.round(Color.blue(color) * 0.8f);
-            return Color.argb(a,
-                    Math.min(r, 255),
-                    Math.min(g, 255),
-                    Math.min(b, 255));
+            return Color.argb(a, Math.min(r, 255), Math.min(g, 255), Math.min(b, 255));
         }
 
-        private void drawText(Canvas canvas, int color, String text, int x, int y, int stroke, int padding) {
+        private void drawText(
+                Canvas canvas, int color, String text, int x, int y, int stroke, int padding) {
             Paint paint = new Paint();
             Paint.FontMetrics metrics = paint.getFontMetrics();
             paint.setStyle(Paint.Style.FILL);
@@ -326,7 +323,9 @@ public class BitmapImageFactory extends ImageFactory {
                 y = 0;
             }
 
-            Bitmap maskedImage = Bitmap.createBitmap(probDist.length, probDist[0].length, Bitmap.Config.ARGB_8888);
+            Bitmap maskedImage =
+                    Bitmap.createBitmap(
+                            probDist.length, probDist[0].length, Bitmap.Config.ARGB_8888);
             for (int xCor = 0; xCor < probDist.length; xCor++) {
                 for (int yCor = 0; yCor < probDist[xCor].length; yCor++) {
                     float opacity = probDist[xCor][yCor];
@@ -339,8 +338,8 @@ public class BitmapImageFactory extends ImageFactory {
                     maskedImage.setPixel(xCor, yCor, darker(Color.argb(opacity, r, g, b)));
                 }
             }
-            Canvas canvas =  new Canvas(image);
-            canvas.drawBitmap(maskedImage, x, y,null);
+            Canvas canvas = new Canvas(image);
+            canvas.drawBitmap(maskedImage, x, y, null);
         }
     }
 }
diff --git a/android/gradlew.bat b/android/gradlew.bat
deleted file mode 120000
index 284013288..000000000
--- a/android/gradlew.bat
+++ /dev/null
@@ -1,1 +0,0 @@
-../gradlew.bat
\ No newline at end of file
diff --git a/android/gradlew.bat b/android/gradlew.bat
new file mode 100644
index 000000000..b6d31956c
--- /dev/null
+++ b/android/gradlew.bat
@@ -0,0 +1,1 @@
+call ..\gradlew.bat
\ No newline at end of file
diff --git a/android/settings.gradle b/android/settings.gradle
index 1406c75cc..543320081 100644
--- a/android/settings.gradle
+++ b/android/settings.gradle
@@ -1,3 +1,3 @@
 rootProject.name='android'
-include ':pytorch-native'
 include ':core'
+include ':pytorch-native'
diff --git a/api/src/main/java/ai/djl/Model.java b/api/src/main/java/ai/djl/Model.java
index 95b9866d9..0fec77426 100644
--- a/api/src/main/java/ai/djl/Model.java
+++ b/api/src/main/java/ai/djl/Model.java
@@ -217,6 +217,7 @@ public interface Model extends AutoCloseable {
      * @return the {@link NDManager}
      */
     NDManager getNDManager();
+
     /**
      * Creates a new {@link Trainer} instance for a Model.
      *
diff --git a/api/src/main/java/ai/djl/translate/DeferredTranslatorFactory.java b/api/src/main/java/ai/djl/translate/DeferredTranslatorFactory.java
new file mode 100644
index 000000000..c500cebf3
--- /dev/null
+++ b/api/src/main/java/ai/djl/translate/DeferredTranslatorFactory.java
@@ -0,0 +1,83 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.translate;
+
+import ai.djl.Model;
+import ai.djl.repository.zoo.Criteria;
+import ai.djl.util.Pair;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.Constructor;
+import java.lang.reflect.Type;
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * A {@link TranslatorFactory} that creates the {@link Translator} based on serving.properties file.
+ *
+ * <p>The {@link Criteria} API cannot access serving.properties files before it's downloaded. A
+ * {@code DeferredTranslatorFactory} assumes serving.properties will provide proper {@link
+ * Translator}. If no translatorFactory is provided in serving.properties, a {@link
+ * TranslateException} will be thrown.
+ */
+public class DeferredTranslatorFactory implements TranslatorFactory {
+
+    private static final Logger logger = LoggerFactory.getLogger(DeferredTranslatorFactory.class);
+
+    /** {@inheritDoc} */
+    @Override
+    public Set<Pair<Type, Type>> getSupportedTypes() {
+        return Collections.emptySet();
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public boolean isSupported(Class<?> input, Class<?> output) {
+        return true;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public Translator<?, ?> newInstance(
+            Class<?> input, Class<?> output, Model model, Map<String, ?> arguments)
+            throws TranslateException {
+        String factoryClass = ArgumentsUtil.stringValue(arguments, "translatorFactory");
+        if (factoryClass == null || factoryClass.isEmpty()) {
+            throw new TranslateException("No translatorFactory defined.");
+        }
+
+        TranslatorFactory factory = loadTranslatorFactory(factoryClass);
+        if (factory == null) {
+            throw new TranslateException("Failed to load translatorFactory: " + factoryClass);
+        } else if (!factory.isSupported(input, output)) {
+            throw new TranslateException(factoryClass + " doesn't support Input/Output.");
+        }
+        logger.info("Using TranslatorFactory: {}", factoryClass);
+        return factory.newInstance(input, output, model, arguments);
+    }
+
+    private TranslatorFactory loadTranslatorFactory(String className) {
+        try {
+            Class<?> clazz = Class.forName(className);
+            Class<? extends TranslatorFactory> subclass = clazz.asSubclass(TranslatorFactory.class);
+            Constructor<? extends TranslatorFactory> constructor = subclass.getConstructor();
+            return constructor.newInstance();
+        } catch (Throwable e) {
+            logger.trace("Not able to load TranslatorFactory: " + className, e);
+        }
+        return null;
+    }
+}
diff --git a/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java b/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
index 4f768f1b8..0755a9688 100644
--- a/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
+++ b/api/src/main/java/ai/djl/translate/ServingTranslatorFactory.java
@@ -49,7 +49,7 @@ import java.util.stream.Stream;
 import javax.tools.JavaCompiler;
 import javax.tools.ToolProvider;
 
-/** A {@link TranslatorFactory} that creates an generic {@link Translator}. */
+/** A {@link TranslatorFactory} that creates a generic {@link Translator}. */
 public class ServingTranslatorFactory implements TranslatorFactory {
 
     private static final Logger logger = LoggerFactory.getLogger(ServingTranslatorFactory.class);
diff --git a/engines/onnxruntime/onnxruntime-android/build.gradle b/engines/onnxruntime/onnxruntime-android/build.gradle
new file mode 100644
index 000000000..c14eaf21f
--- /dev/null
+++ b/engines/onnxruntime/onnxruntime-android/build.gradle
@@ -0,0 +1,21 @@
+group "ai.djl.android"
+
+dependencies {
+    api(project(":engines:onnxruntime:onnxruntime-engine")) {
+        exclude group: "com.microsoft.onnxruntime", module: "onnxruntime"
+    }
+    api "com.microsoft.onnxruntime:onnxruntime-android:${onnxruntime_version}"
+}
+
+publishing {
+    publications {
+        maven(MavenPublication) {
+            artifactId "onnxruntime"
+            pom {
+                name = "DJL ONNX Runtime engine for Android"
+                description = "Deep Java Library (DJL) Engine Adapter for ONNX Runtime"
+                url = "http://www.djl.ai/engines/onnxruntime/${project.name}"
+            }
+        }
+    }
+}
diff --git a/engines/onnxruntime/onnxruntime-android/gradlew b/engines/onnxruntime/onnxruntime-android/gradlew
new file mode 120000
index 000000000..ab9334b00
--- /dev/null
+++ b/engines/onnxruntime/onnxruntime-android/gradlew
@@ -0,0 +1,1 @@
+../../../gradlew
\ No newline at end of file
diff --git a/engines/onnxruntime/onnxruntime-android/src/main/.empty b/engines/onnxruntime/onnxruntime-android/src/main/.empty
new file mode 100644
index 000000000..e69de29bb
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtEngine.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtEngine.java
index c590524f6..b48da766c 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtEngine.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtEngine.java
@@ -51,6 +51,7 @@ public final class PtEngine extends Engine {
     static Engine newInstance() {
         try {
             LibUtils.loadLibrary();
+            JniUtils.setGradMode(false);
             if (Integer.getInteger("ai.djl.pytorch.num_interop_threads") != null) {
                 JniUtils.setNumInteropThreads(
                         Integer.getInteger("ai.djl.pytorch.num_interop_threads"));
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtGradientCollector.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtGradientCollector.java
index a200b2c40..fad7755fa 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtGradientCollector.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtGradientCollector.java
@@ -20,6 +20,14 @@ import ai.djl.training.GradientCollector;
 /** {@code PtGradientCollector} is the PyTorch implementation of {@link GradientCollector}. */
 public class PtGradientCollector implements GradientCollector {
 
+    private boolean gradModel;
+
+    /** Constructs a new {@code PtGradientCollector} instance. */
+    public PtGradientCollector() {
+        gradModel = JniUtils.isGradMode();
+        JniUtils.setGradMode(true);
+    }
+
     /** {@inheritDoc} */
     @Override
     public void backward(NDArray target) {
@@ -49,6 +57,9 @@ public class PtGradientCollector implements GradientCollector {
     /** {@inheritDoc} */
     @Override
     public void close() {
+        if (!gradModel) {
+            JniUtils.setGradMode(false);
+        }
         // TODO: do some clean up if necessary
     }
 }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
index e6c128728..01cb98991 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
@@ -1055,10 +1055,13 @@ public class PtNDArray extends NativeResource<Long> implements NDArray {
     /** {@inheritDoc} */
     @Override
     public PtNDArray argSort(int axis, boolean ascending) {
-        if (!ascending) {
-            throw new UnsupportedOperationException("Only support ascending!");
+        PtNDArray arr = JniUtils.argSort(this, axis, false);
+        if (ascending) {
+            return arr;
         }
-        return JniUtils.argSort(this, axis, false);
+        PtNDArray flip = JniUtils.flip(arr, new long[] {axis});
+        arr.close();
+        return flip;
     }
 
     /** {@inheritDoc} */
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
index 7e6a3ff41..bf7106b6c 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
@@ -82,6 +82,14 @@ public final class JniUtils {
         }
     }
 
+    public static boolean isGradMode() {
+        return PyTorchLibrary.LIB.torchIsGradMode();
+    }
+
+    public static void setGradMode(boolean enable) {
+        PyTorchLibrary.LIB.torchSetGradMode(enable);
+    }
+
     public static int getNumInteropThreads() {
         return PyTorchLibrary.LIB.torchGetNumInteropThreads();
     }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
index c3fcc64a5..10da7843d 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/PyTorchLibrary.java
@@ -24,6 +24,10 @@ final class PyTorchLibrary {
 
     private PyTorchLibrary() {}
 
+    native boolean torchIsGradMode();
+
+    native void torchSetGradMode(boolean enable);
+
     native int torchGetNumInteropThreads();
 
     native int torchGetNumThreads();
diff --git a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_system.cc b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_system.cc
index 736eea1fc..4f9b17b88 100644
--- a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_system.cc
+++ b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_system.cc
@@ -39,6 +39,19 @@ using namespace torch::autograd::profiler;
 
 // The file is the implementation for PyTorch system-wide operations
 
+JNIEXPORT jboolean JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchIsGradMode(JNIEnv* env, jobject jthis) {
+  API_BEGIN()
+  return c10::GradMode::is_enabled();
+  API_END_RETURN()
+}
+
+JNIEXPORT void JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchSetGradMode(
+    JNIEnv* env, jobject jthis, jboolean enable) {
+  API_BEGIN()
+  c10::GradMode::set_enabled(enable);
+  API_END()
+}
+
 JNIEXPORT jint JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchGetNumInteropThreads(JNIEnv* env, jobject jthis) {
   API_BEGIN()
   return torch::get_num_interop_threads();
diff --git a/extensions/tokenizers/rust/src/lib.rs b/extensions/tokenizers/rust/src/lib.rs
index 3b0d52a7e..17bfa79b8 100644
--- a/extensions/tokenizers/rust/src/lib.rs
+++ b/extensions/tokenizers/rust/src/lib.rs
@@ -18,6 +18,8 @@ use std::str::FromStr;
 use tk::tokenizer::{EncodeInput, Encoding};
 use tk::Tokenizer;
 use tk::{FromPretrainedParameters, Offsets};
+use tk::utils::truncation::{TruncationParams, TruncationStrategy};
+use tk::utils::padding::{PaddingParams, PaddingStrategy};
 
 use jni::objects::{JClass, JMethodID, JObject, JString, JValue, ReleaseMode};
 use jni::sys::{
@@ -405,6 +407,81 @@ pub extern "system" fn Java_ai_djl_huggingface_tokenizers_jni_TokenizersLibrary_
     ret
 }
 
+#[no_mangle]
+pub extern "system" fn Java_ai_djl_huggingface_tokenizers_jni_TokenizersLibrary_setPadding(
+    env: JNIEnv,
+    _: JObject,
+    handle: jlong,
+    max_length: jlong,
+    padding_strategy: JString,
+    pad_to_multiple_of: jlong,
+) {
+    let strategy: String = env
+        .get_string(padding_strategy)
+        .expect("Couldn't get java string!")
+        .into();
+    let len = max_length as usize;
+    let res_strategy = match strategy.as_ref() {
+        "batch_longest" => Ok(PaddingStrategy::BatchLongest),
+        "fixed" => Ok(PaddingStrategy::Fixed(len)),
+        _ => Err("strategy must be one of [batch_longest, fixed]"),
+    };
+
+    let mut params = PaddingParams::default();
+    params.strategy = res_strategy.unwrap();
+    params.pad_to_multiple_of = Some(pad_to_multiple_of as usize);
+    let tokenizer = cast_handle::<Tokenizer>(handle);
+    tokenizer.with_padding(Some(params));
+}
+
+#[no_mangle]
+pub extern "system" fn Java_ai_djl_huggingface_tokenizers_jni_TokenizersLibrary_disablePadding(
+    _env: JNIEnv,
+    _: JObject,
+    handle: jlong,
+) {
+    let tokenizer = cast_handle::<Tokenizer>(handle);
+    tokenizer.with_padding(None);
+}
+
+#[no_mangle]
+pub extern "system" fn Java_ai_djl_huggingface_tokenizers_jni_TokenizersLibrary_setTruncation(
+    env: JNIEnv,
+    _: JObject,
+    handle: jlong,
+    truncation_max_length: jlong,
+    truncation_strategy: JString,
+    truncation_stride: jlong,
+) {
+    let strategy: String = env
+        .get_string(truncation_strategy)
+        .expect("Couldn't get java string!")
+        .into();
+    let res_strategy = match strategy.as_ref() {
+       "longest_first" => Ok(TruncationStrategy::LongestFirst),
+       "only_first" => Ok(TruncationStrategy::OnlyFirst),
+       "only_second" => Ok(TruncationStrategy::OnlySecond),
+       _ => Err("strategy must be one of [longest_first, only_first, only_second]"),
+    };
+    let mut params = TruncationParams::default();
+    params.max_length = truncation_max_length as usize;
+    params.strategy = res_strategy.unwrap();
+    params.stride = truncation_stride as usize;
+
+    let tokenizer = cast_handle::<Tokenizer>(handle);
+    tokenizer.with_truncation(Some(params));
+}
+
+#[no_mangle]
+pub extern "system" fn Java_ai_djl_huggingface_tokenizers_jni_TokenizersLibrary_disableTruncation(
+    _env: JNIEnv,
+    _: JObject,
+    handle: jlong,
+) {
+    let tokenizer = cast_handle::<Tokenizer>(handle);
+    tokenizer.with_truncation(None);
+}
+
 fn to_handle<T: 'static>(val: T) -> jlong {
     let handle = Box::into_raw(Box::new(val)) as jlong;
     handle
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizer.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizer.java
index 123ba7c84..f80779960 100644
--- a/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizer.java
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizer.java
@@ -302,6 +302,90 @@ public final class HuggingFaceTokenizer extends NativeResource<Long> implements
         return decode(ids, !addSpecialTokens);
     }
 
+    /**
+     * Sets the truncation and padding behavior for the tokenizer.
+     *
+     * @param truncationStrategy the {@code TruncationStrategy} to use
+     * @param paddingStrategy the {@code PaddingStrategy} to use
+     * @param maxLength the maximum length to pad/truncate sequences to
+     */
+    public void setTruncationAndPadding(
+            TruncationStrategy truncationStrategy,
+            PaddingStrategy paddingStrategy,
+            long maxLength) {
+        setTruncationAndPadding(truncationStrategy, paddingStrategy, maxLength, 0, 0);
+    }
+
+    /**
+     * Sets the truncation and padding behavior for the tokenizer.
+     *
+     * @param truncationStrategy the {@code TruncationStrategy} to use
+     * @param paddingStrategy the {@code PaddingStrategy} to use
+     * @param maxLength the maximum length to pad/truncate sequences to
+     * @param stride value to use when handling overflow
+     */
+    public void setTruncationAndPadding(
+            TruncationStrategy truncationStrategy,
+            PaddingStrategy paddingStrategy,
+            long maxLength,
+            long stride) {
+        setTruncationAndPadding(truncationStrategy, paddingStrategy, maxLength, stride, 0);
+    }
+
+    /**
+     * Sets the truncation and padding behavior for the tokenizer.
+     *
+     * @param truncationStrategy the {@code TruncationStrategy} to use
+     * @param paddingStrategy the {@code PaddingStrategy} to use
+     * @param maxLength the maximum length to pad/truncate sequences to
+     * @param stride value to use when handling overflow
+     * @param padToMultipleOf pad sequence length to multiple of value
+     */
+    public void setTruncationAndPadding(
+            TruncationStrategy truncationStrategy,
+            PaddingStrategy paddingStrategy,
+            long maxLength,
+            long stride,
+            long padToMultipleOf) {
+        setTruncation(truncationStrategy, maxLength, stride);
+        setPadding(paddingStrategy, maxLength, padToMultipleOf);
+    }
+
+    /**
+     * Sets the truncation behavior for the tokenizer.
+     *
+     * @param truncationStrategy the {@code TruncationStrategy} to use
+     * @param maxLength the maximum length to truncate sequences to
+     * @param stride value to use when handling overflow
+     */
+    public void setTruncation(TruncationStrategy truncationStrategy, long maxLength, long stride) {
+        if (truncationStrategy == TruncationStrategy.DO_NOT_TRUNCATE) {
+            TokenizersLibrary.LIB.disableTruncation(getHandle());
+        } else {
+            TokenizersLibrary.LIB.setTruncation(
+                    getHandle(), maxLength, truncationStrategy.toString().toLowerCase(), stride);
+        }
+    }
+
+    /**
+     * Sets the padding behavior for the tokenizer.
+     *
+     * @param paddingStrategy the {@code PaddingStrategy} to use
+     * @param maxLength the maximum length to pad sequences to
+     * @param padToMultipleOf pad sequence length to multiple of value
+     */
+    public void setPadding(PaddingStrategy paddingStrategy, long maxLength, long padToMultipleOf) {
+        if (paddingStrategy == PaddingStrategy.DO_NOT_PAD) {
+            TokenizersLibrary.LIB.disablePadding(getHandle());
+        } else {
+            TokenizersLibrary.LIB.setPadding(
+                    getHandle(),
+                    maxLength,
+                    paddingStrategy.toString().toLowerCase(),
+                    padToMultipleOf);
+        }
+    }
+
     private Encoding toEncoding(long encoding) {
         long[] ids = TokenizersLibrary.LIB.getTokenIds(encoding);
         long[] typeIds = TokenizersLibrary.LIB.getTypeIds(encoding);
@@ -315,4 +399,17 @@ public final class HuggingFaceTokenizer extends NativeResource<Long> implements
         return new Encoding(
                 ids, typeIds, tokens, wordIds, attentionMask, specialTokenMask, charSpans);
     }
+
+    enum TruncationStrategy {
+        DO_NOT_TRUNCATE,
+        LONGEST_FIRST,
+        ONLY_FIRST,
+        ONLY_SECOND
+    }
+
+    enum PaddingStrategy {
+        DO_NOT_PAD,
+        BATCH_LONGEST,
+        FIXED
+    }
 }
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/jni/TokenizersLibrary.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/jni/TokenizersLibrary.java
index b532886bf..b0e9095ac 100644
--- a/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/jni/TokenizersLibrary.java
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/tokenizers/jni/TokenizersLibrary.java
@@ -52,4 +52,14 @@ public final class TokenizersLibrary {
     public native CharSpan[] getTokenCharSpans(long encoding);
 
     public native String decode(long tokenizer, long[] ids, boolean addSpecialTokens);
+
+    public native void disablePadding(long tokenizer);
+
+    public native void setPadding(
+            long tokenizer, long maxLength, String paddingStrategy, long padToMultipleOf);
+
+    public native void disableTruncation(long tokenizer);
+
+    public native void setTruncation(
+            long tokenizer, long maxLength, String truncationStrategy, long stride);
 }
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java
new file mode 100644
index 000000000..92586788c
--- /dev/null
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslator.java
@@ -0,0 +1,232 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.huggingface.translator;
+
+import ai.djl.huggingface.tokenizers.Encoding;
+import ai.djl.huggingface.tokenizers.HuggingFaceTokenizer;
+import ai.djl.modality.Classifications;
+import ai.djl.ndarray.NDArray;
+import ai.djl.ndarray.NDList;
+import ai.djl.ndarray.NDManager;
+import ai.djl.translate.ArgumentsUtil;
+import ai.djl.translate.Batchifier;
+import ai.djl.translate.TranslateException;
+import ai.djl.translate.Translator;
+import ai.djl.translate.TranslatorContext;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+/** The translator for Huggingface fill mask model. */
+public class FillMaskTranslator implements Translator<String, Classifications> {
+
+    private HuggingFaceTokenizer tokenizer;
+    private String maskToken;
+    private long maskTokenId;
+    private int topK;
+    private Batchifier batchifier;
+
+    FillMaskTranslator(
+            HuggingFaceTokenizer tokenizer, String maskToken, int topK, Batchifier batchifier) {
+        this.tokenizer = tokenizer;
+        this.maskToken = maskToken;
+        this.topK = topK;
+        this.batchifier = batchifier;
+        Encoding encoding = tokenizer.encode(maskToken, false);
+        maskTokenId = encoding.getIds()[0];
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public Batchifier getBatchifier() {
+        return batchifier;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public NDList processInput(TranslatorContext ctx, String input) throws TranslateException {
+        NDManager manager = ctx.getNDManager();
+        Encoding encoding = tokenizer.encode(input);
+        long[] indices = encoding.getIds();
+        int maskIndex = -1;
+        for (int i = 0; i < indices.length; ++i) {
+            if (indices[i] == maskTokenId) {
+                if (maskIndex != -1) {
+                    throw new TranslateException("Only one mask supported.");
+                }
+                maskIndex = i;
+            }
+        }
+        if (maskIndex == -1) {
+            throw new TranslateException("Mask token " + maskToken + " not found.");
+        }
+        ctx.setAttachment("maskIndex", maskIndex);
+        long[] attentionMask = encoding.getAttentionMask();
+        NDList ndList = new NDList(2);
+        ndList.add(manager.create(indices));
+        ndList.add(manager.create(attentionMask));
+        return ndList;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public Classifications processOutput(TranslatorContext ctx, NDList list) {
+        int maskIndex = (int) ctx.getAttachment("maskIndex");
+        NDArray prob = list.get(0).get(maskIndex).softmax(0);
+        NDArray array = prob.argSort(0, false);
+        long[] classIds = new long[topK];
+        List<Double> probabilities = new ArrayList<>(topK);
+        for (int i = 0; i < topK; ++i) {
+            classIds[i] = array.getLong(i);
+            probabilities.add((double) prob.getFloat(classIds[i]));
+        }
+        String[] classes = tokenizer.decode(classIds).trim().split(" ");
+        return new Classifications(Arrays.asList(classes), probabilities);
+    }
+
+    /**
+     * Creates a builder to build a {@code FillMaskTranslator}.
+     *
+     * @return a new builder
+     */
+    public static Builder builder() {
+        return new Builder();
+    }
+
+    /**
+     * Creates a builder to build a {@code FillMaskTranslator}.
+     *
+     * @param arguments the models' arguments
+     * @return a new builder
+     */
+    public static Builder builder(Map<String, ?> arguments) {
+        Builder builder = builder();
+        builder.configure(arguments);
+
+        return builder;
+    }
+
+    /** The builder for fill mask translator. */
+    public static final class Builder {
+
+        private String maskedToken = "[MASK]";
+        private int topK = 5;
+        private String tokenizerName;
+        private Path tokenizerPath;
+        private boolean addSpecialTokens = true;
+        private Batchifier batchifier = Batchifier.STACK;
+
+        /**
+         * Sets the id of the mask {@link Translator}.
+         *
+         * @param maskedToken the id of the mask
+         * @return this builder
+         */
+        public Builder optMaskToken(String maskedToken) {
+            this.maskedToken = maskedToken;
+            return this;
+        }
+
+        /**
+         * Set the topK number of classes to be displayed.
+         *
+         * @param topK the number of top classes to return
+         * @return this builder
+         */
+        public Builder optTopK(int topK) {
+            this.topK = topK;
+            return this;
+        }
+
+        /**
+         * Sets the name of the tokenizer for the {@link Translator}.
+         *
+         * @param tokenizerName the name of the tokenizer
+         * @return this builder
+         */
+        public Builder optTokenizerName(String tokenizerName) {
+            this.tokenizerName = tokenizerName;
+            return this;
+        }
+
+        /**
+         * Sets the file path of the tokenizer for the {@link Translator}.
+         *
+         * @param tokenizerPath the path of the tokenizer
+         * @return this builder
+         */
+        public Builder optTokenizerPath(Path tokenizerPath) {
+            this.tokenizerPath = tokenizerPath;
+            return this;
+        }
+
+        /**
+         * Sets if add special tokens for the {@link Translator}.
+         *
+         * @param addSpecialTokens true to add special tokens
+         * @return this builder
+         */
+        public Builder optAddSpecialTokens(boolean addSpecialTokens) {
+            this.addSpecialTokens = addSpecialTokens;
+            return this;
+        }
+
+        /**
+         * Sets the {@link Batchifier} for the {@link Translator}.
+         *
+         * @param batchifier true to include token types
+         * @return this builder
+         */
+        public Builder optBatchifier(Batchifier batchifier) {
+            this.batchifier = batchifier;
+            return this;
+        }
+
+        /**
+         * Configures the builder with the model arguments.
+         *
+         * @param arguments the model arguments
+         */
+        public void configure(Map<String, ?> arguments) {
+            optMaskToken(ArgumentsUtil.stringValue(arguments, "maskToken", "[MASK]"));
+            optTopK(ArgumentsUtil.intValue(arguments, "topK", 5));
+            optTokenizerName(ArgumentsUtil.stringValue(arguments, "tokenizer"));
+            optAddSpecialTokens(ArgumentsUtil.booleanValue(arguments, "addSpecialTokens", true));
+            String batchifierStr = ArgumentsUtil.stringValue(arguments, "batchifier", "stack");
+            optBatchifier(Batchifier.fromString(batchifierStr));
+        }
+
+        /**
+         * Builds the translator.
+         *
+         * @return the new translator
+         * @throws IOException if I/O error occurs
+         */
+        public FillMaskTranslator build() throws IOException {
+            HuggingFaceTokenizer tokenizer;
+            Map<String, String> options = new ConcurrentHashMap<>();
+            options.put("addSpecialTokens", String.valueOf(addSpecialTokens));
+            if (tokenizerName != null) {
+                tokenizer = HuggingFaceTokenizer.newInstance(tokenizerName, options);
+            } else {
+                tokenizer = HuggingFaceTokenizer.newInstance(tokenizerPath, options);
+            }
+            return new FillMaskTranslator(tokenizer, maskedToken, topK, batchifier);
+        }
+    }
+}
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslatorFactory.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslatorFactory.java
new file mode 100644
index 000000000..d3db2dab1
--- /dev/null
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/FillMaskTranslatorFactory.java
@@ -0,0 +1,67 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.huggingface.translator;
+
+import ai.djl.Model;
+import ai.djl.modality.Classifications;
+import ai.djl.modality.Input;
+import ai.djl.modality.Output;
+import ai.djl.modality.nlp.translator.TextClassificationServingTranslator;
+import ai.djl.translate.TranslateException;
+import ai.djl.translate.Translator;
+import ai.djl.translate.TranslatorFactory;
+import ai.djl.util.Pair;
+
+import java.io.IOException;
+import java.lang.reflect.Type;
+import java.nio.file.Path;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/** A {@link TranslatorFactory} that creates a {@link FillMaskTranslator} instance. */
+public class FillMaskTranslatorFactory implements TranslatorFactory {
+
+    private static final Set<Pair<Type, Type>> SUPPORTED_TYPES = new HashSet<>();
+
+    static {
+        SUPPORTED_TYPES.add(new Pair<>(String.class, Classifications.class));
+        SUPPORTED_TYPES.add(new Pair<>(Input.class, Output.class));
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public Set<Pair<Type, Type>> getSupportedTypes() {
+        return SUPPORTED_TYPES;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public Translator<?, ?> newInstance(
+            Class<?> input, Class<?> output, Model model, Map<String, ?> arguments)
+            throws TranslateException {
+        Path modelPath = model.getModelPath();
+        try {
+            FillMaskTranslator translator =
+                    FillMaskTranslator.builder(arguments).optTokenizerPath(modelPath).build();
+            if (input == String.class && output == Classifications.class) {
+                return translator;
+            } else if (input == Input.class && output == Output.class) {
+                return new TextClassificationServingTranslator(translator);
+            }
+            throw new IllegalArgumentException("Unsupported input/output types.");
+        } catch (IOException e) {
+            throw new TranslateException("Failed to load tokenizer.", e);
+        }
+    }
+}
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/QuestionAnsweringTranslator.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/QuestionAnsweringTranslator.java
index ebc3798bd..5dcc7ccab 100644
--- a/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/QuestionAnsweringTranslator.java
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/QuestionAnsweringTranslator.java
@@ -18,6 +18,7 @@ import ai.djl.modality.nlp.qa.QAInput;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
+import ai.djl.ndarray.index.NDIndex;
 import ai.djl.translate.ArgumentsUtil;
 import ai.djl.translate.Batchifier;
 import ai.djl.translate.Translator;
@@ -25,8 +26,6 @@ import ai.djl.translate.TranslatorContext;
 
 import java.io.IOException;
 import java.nio.file.Path;
-import java.util.Arrays;
-import java.util.List;
 import java.util.Map;
 import java.util.concurrent.ConcurrentHashMap;
 
@@ -55,8 +54,7 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
     public NDList processInput(TranslatorContext ctx, QAInput input) {
         NDManager manager = ctx.getNDManager();
         Encoding encoding = tokenizer.encode(input.getQuestion(), input.getParagraph());
-        String[] tokens = encoding.getTokens();
-        ctx.setAttachment("tokens", Arrays.asList(tokens));
+        ctx.setAttachment("encoding", encoding);
         long[] indices = encoding.getIds();
         long[] attentionMask = encoding.getAttentionMask();
         NDList ndList = new NDList(3);
@@ -71,10 +69,14 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
 
     /** {@inheritDoc} */
     @Override
-    @SuppressWarnings("unchecked")
     public String processOutput(TranslatorContext ctx, NDList list) {
-        NDArray startLogits = list.get(0);
-        NDArray endLogits = list.get(1);
+        // PyTorch InferenceMode tensor is read only, must clone it
+        NDArray startLogits = list.get(0).duplicate();
+        NDArray endLogits = list.get(1).duplicate();
+
+        // exclude <CLS>, TODO: exclude impossible ids properly and handle max answer length
+        startLogits.set(new NDIndex(0), 0);
+        endLogits.set(new NDIndex(0), 0);
         int startIdx = (int) startLogits.argMax().getLong();
         int endIdx = (int) endLogits.argMax().getLong();
         if (startIdx > endIdx) {
@@ -82,8 +84,12 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
             startIdx = endIdx;
             endIdx = tmp;
         }
-        List<String> tokens = (List<String>) ctx.getAttachment("tokens");
-        return tokenizer.buildSentence(tokens.subList(startIdx, endIdx + 1));
+        Encoding encoding = (Encoding) ctx.getAttachment("encoding");
+        long[] indices = encoding.getIds();
+        int len = endIdx - startIdx + 1;
+        long[] ids = new long[len];
+        System.arraycopy(indices, startIdx, ids, 0, len);
+        return tokenizer.decode(ids).trim();
     }
 
     /**
@@ -131,7 +137,7 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
         /**
          * Sets the file path of the tokenizer for the {@link Translator}.
          *
-         * @param tokenizerPath the name of the tokenizer
+         * @param tokenizerPath the path of the tokenizer
          * @return this builder
          */
         public Builder optTokenizerPath(Path tokenizerPath) {
@@ -167,7 +173,7 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
          * @param batchifier true to include token types
          * @return this builder
          */
-        public Builder optBatchifer(Batchifier batchifier) {
+        public Builder optBatchifier(Batchifier batchifier) {
             this.batchifier = batchifier;
             return this;
         }
@@ -181,9 +187,8 @@ public class QuestionAnsweringTranslator implements Translator<QAInput, String>
             optTokenizerName(ArgumentsUtil.stringValue(arguments, "tokenizer"));
             optIncludeTokenTypes(ArgumentsUtil.booleanValue(arguments, "includeTokenTypes"));
             optAddSpecialTokens(ArgumentsUtil.booleanValue(arguments, "addSpecialTokens", true));
-            optBatchifer(
-                    Batchifier.fromString(
-                            ArgumentsUtil.stringValue(arguments, "batchifier", "stack")));
+            String batchifierStr = ArgumentsUtil.stringValue(arguments, "batchifier", "stack");
+            optBatchifier(Batchifier.fromString(batchifierStr));
         }
 
         /**
diff --git a/extensions/tokenizers/src/main/python/.gitignore b/extensions/tokenizers/src/main/python/.gitignore
index bee8a64b7..a63c8dced 100644
--- a/extensions/tokenizers/src/main/python/.gitignore
+++ b/extensions/tokenizers/src/main/python/.gitignore
@@ -1,1 +1,3 @@
 __pycache__
+model/
+processed_models.json
diff --git a/extensions/tokenizers/src/main/python/huggingface_converter.py b/extensions/tokenizers/src/main/python/huggingface_converter.py
index 7727993d4..9023d8553 100644
--- a/extensions/tokenizers/src/main/python/huggingface_converter.py
+++ b/extensions/tokenizers/src/main/python/huggingface_converter.py
@@ -115,6 +115,9 @@ class HuggingfaceConverter:
                 continue
 
             result, reason, size = self.save_model(model_id)
+            if not result:
+                logging.error(reason)
+
             self.save_progress(model_id, model_info.sha, result, reason, size)
             shutil.rmtree(temp_dir)
 
@@ -139,10 +142,12 @@ class HuggingfaceConverter:
         if not model_file:
             return False, "Failed to trace model", -1
 
-        if not self.verify_jit_model(hf_pipeline, task, model_file):
-            return False, "Failed verify jit model", -1
+        result, reason = self.verify_jit_model(hf_pipeline, task, model_file)
+        if not result:
+            return False, reason, -1
 
-        size = self.save_model_zoo(task, model_id, temp_dir)
+        size = self.save_model_zoo(task, model_id, temp_dir,
+                                   hf_pipeline.tokenizer.mask_token)
 
         return True, None, size
 
@@ -179,7 +184,8 @@ class HuggingfaceConverter:
         script_module.save(model_file)
         return model_file
 
-    def save_model_zoo(self, task: str, model_id: str, temp_dir: str):
+    def save_model_zoo(self, task: str, model_id: str, temp_dir: str,
+                       mask_token: str):
         artifact_ids = model_id.split("/")
         model_name = artifact_ids[-1]
 
@@ -193,9 +199,11 @@ class HuggingfaceConverter:
         # Save serving.properties
         serving_file = os.path.join(temp_dir, "serving.properties")
         with open(serving_file, 'w') as f:
-            f.write(
-                f"engine=PyTorch\noption.modelName={model_name}\noption.mapLocation=true\n"
-                f"translatorFactory={translator}")
+            f.write(f"engine=PyTorch\n"
+                    f"option.modelName={model_name}\n"
+                    f"option.mapLocation=true\n"
+                    f"maskToken={mask_token}\n"
+                    f"translatorFactory={translator}")
 
         # Save model as .zip file
         zip_file = os.path.join(model_dir, f"{model_name}.zip")
@@ -232,12 +240,16 @@ class HuggingfaceConverter:
         tokenizer = hf_pipeline.tokenizer
 
         if task == "question-answering":
-            answer_start = torch.argmax(out["start_logits"])
-            answer_end = torch.argmax(out["end_logits"]) + 1
+            start_ = out["start_logits"]
+            end_ = out["end_logits"]
+            start_[0, 0] = 0
+            end_[0, 0] = 0
+            answer_start = torch.argmax(start_)
+            answer_end = torch.argmax(end_) + 1
 
             out_ids = input_ids[0].tolist()[answer_start:answer_end]
             tokens = tokenizer.convert_ids_to_tokens(out_ids)
-            prediction = tokenizer.convert_tokens_to_string(tokens)
+            prediction = tokenizer.convert_tokens_to_string(tokens).strip()
 
             if prediction.lower() != expected_output:
                 inputs = SUPPORTED_TASK[task]["inputs"]
@@ -249,11 +261,10 @@ class HuggingfaceConverter:
                         f"pipeline output differs from expected: {pipeline_output}"
                     )
         elif task == "fill-mask":
-            masked_token_id = tokenizer.mask_token_id
-            masked_index = torch.nonzero(
-                input_ids.squeeze(0) == masked_token_id,
-                as_tuple=False).squeeze(0)
-            logits = out['logits'][0, masked_index]
+            mask_token_id = tokenizer.mask_token_id
+            mask_index = torch.nonzero(input_ids.squeeze(0) == mask_token_id,
+                                       as_tuple=False).squeeze(0)
+            logits = out['logits'][0, mask_index]
             answer = torch.argmax(logits)
             prediction = tokenizer.decode(answer).strip()
 
@@ -273,7 +284,7 @@ class HuggingfaceConverter:
             if not hasattr(out, "last_hidden_layer"):
                 return False, f"Unexpected inference result: {out}"
 
-        return True
+        return True, None
 
     @staticmethod
     def load_model(task: str, model_id: str):
diff --git a/settings.gradle b/settings.gradle
index 412daf4fe..ba0569d62 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -9,6 +9,7 @@ include ':engines:mxnet:jnarator'
 include ':engines:mxnet:mxnet-engine'
 include ':engines:mxnet:mxnet-model-zoo'
 include ':engines:mxnet:native'
+include ':engines:onnxruntime:onnxruntime-android'
 include ':engines:onnxruntime:onnxruntime-engine'
 include ':engines:paddlepaddle:paddlepaddle-engine'
 include ':engines:paddlepaddle:paddlepaddle-model-zoo'
diff --git a/tools/gradle/android-formatter.gradle b/tools/gradle/android-formatter.gradle
new file mode 100644
index 000000000..7ffe40638
--- /dev/null
+++ b/tools/gradle/android-formatter.gradle
@@ -0,0 +1,40 @@
+buildscript {
+    repositories {
+        mavenCentral()
+        maven {
+            url "https://plugins.gradle.org/m2/"
+        }
+    }
+    dependencies {
+        classpath 'com.google.googlejavaformat:google-java-format:1.15.0'
+    }
+}
+
+apply plugin: JavaFormatterPlugin
+
+import com.google.googlejavaformat.java.Formatter
+import com.google.googlejavaformat.java.ImportOrderer
+import com.google.googlejavaformat.java.JavaFormatterOptions
+import com.google.googlejavaformat.java.Main
+import com.google.googlejavaformat.java.RemoveUnusedImports
+
+class JavaFormatterPlugin implements Plugin<Project> {
+    void apply(Project project) {
+        project.task('formatJava') {
+            doLast {
+                if (project.getRootProject() == project) {
+                    return
+                }
+                Main formatter = new Main(new PrintWriter(System.out, true), new PrintWriter(System.err, true), System.in)
+                FileTree dirs = project.fileTree("${project.getProjectDir()}/src").matching {
+                    include "**/*.java"
+                }
+                for (file in dirs) {
+                    if (formatter.format("-a", "-i", file.getAbsolutePath()) != 0) {
+                        throw new GradleException("Format java failed: " + file.getAbsolutePath())
+                    }
+                }
+            }
+        }
+    }
+}
diff --git a/tools/gradle/publish.gradle b/tools/gradle/publish.gradle
index 1c02921e1..2e456bb5f 100644
--- a/tools/gradle/publish.gradle
+++ b/tools/gradle/publish.gradle
@@ -5,6 +5,7 @@ configure([
         project(':engines:ml:xgboost'),
         project(':engines:mxnet:mxnet-engine'),
         project(':engines:mxnet:mxnet-model-zoo'),
+        project(':engines:onnxruntime:onnxruntime-android'),
         project(':engines:onnxruntime:onnxruntime-engine'),
         project(':engines:paddlepaddle:paddlepaddle-engine'),
         project(':engines:paddlepaddle:paddlepaddle-model-zoo'),


Test patch:
diff --git a/android/core/src/androidTest/java/ai/djl/android/core/BitmapWrapperTest.java b/android/core/src/androidTest/java/ai/djl/android/core/BitmapWrapperTest.java
index d54f8dcd9..7b8f005a8 100644
--- a/android/core/src/androidTest/java/ai/djl/android/core/BitmapWrapperTest.java
+++ b/android/core/src/androidTest/java/ai/djl/android/core/BitmapWrapperTest.java
@@ -12,14 +12,13 @@
  */
 package ai.djl.android.core;
 
-import android.content.Context;
+import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation;
 
-import androidx.test.ext.junit.runners.AndroidJUnit4;
+import static org.junit.Assert.assertEquals;
 
-import org.junit.Test;
-import org.junit.runner.RunWith;
+import android.content.Context;
 
-import java.io.IOException;
+import androidx.test.ext.junit.runners.AndroidJUnit4;
 
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.ImageFactory;
@@ -28,8 +27,10 @@ import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 
-import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation;
-import static org.junit.Assert.assertEquals;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import java.io.IOException;
 
 /**
  * Instrumented test, which will execute on an Android device.
@@ -51,7 +52,9 @@ public class BitmapWrapperTest {
     public void testImageToNDArray() throws IOException {
         try (NDManager manager = NDManager.newBaseManager()) {
             ImageFactory factory = ImageFactory.getInstance();
-            Image img = factory.fromUrl("https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
+            Image img =
+                    factory.fromUrl(
+                            "https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
             NDArray array = img.toNDArray(manager);
             assertEquals(new Shape(img.getHeight(), img.getWidth(), 3), array.getShape());
         }
@@ -61,7 +64,9 @@ public class BitmapWrapperTest {
     public void testImageGetSubImage() throws IOException {
         try (NDManager manager = NDManager.newBaseManager()) {
             ImageFactory factory = ImageFactory.getInstance();
-            Image img = factory.fromUrl("https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
+            Image img =
+                    factory.fromUrl(
+                            "https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
             NDArray array = img.toNDArray(manager);
             Image subImg = img.getSubImage(10, 20, 30, 40);
             NDArray subArray = subImg.toNDArray(manager);
@@ -73,7 +78,9 @@ public class BitmapWrapperTest {
     public void testImageDuplicate() throws IOException {
         try (NDManager manager = NDManager.newBaseManager()) {
             ImageFactory factory = ImageFactory.getInstance();
-            Image img = factory.fromUrl("https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
+            Image img =
+                    factory.fromUrl(
+                            "https://github.com/deepjavalibrary/djl/raw/master/examples/src/test/resources/dog_bike_car.jpg");
             NDArray array = img.toNDArray(manager);
             NDArray duplicate = img.duplicate().toNDArray(manager);
             assertEquals(array, duplicate);
@@ -86,9 +93,8 @@ public class BitmapWrapperTest {
             NDArray array = manager.arange(0.0f, 12.0f).reshape(3, 2, 2);
             ImageFactory factory = ImageFactory.getInstance();
             Image image = factory.fromNDArray(array);
-            NDArray converted = image.toNDArray(manager)
-                    .transpose(2, 0, 1)
-                    .toType(DataType.FLOAT32, true);
+            NDArray converted =
+                    image.toNDArray(manager).transpose(2, 0, 1).toType(DataType.FLOAT32, true);
             assertEquals(array, converted);
 
             array = manager.arange(0, 12, 1, DataType.UINT8).reshape(3, 2, 2);
diff --git a/android/core/src/androidTest/java/ai/djl/android/core/ModelLoadingTest.java b/android/core/src/androidTest/java/ai/djl/android/core/ModelLoadingTest.java
index 73277e972..1a0f3387d 100644
--- a/android/core/src/androidTest/java/ai/djl/android/core/ModelLoadingTest.java
+++ b/android/core/src/androidTest/java/ai/djl/android/core/ModelLoadingTest.java
@@ -12,11 +12,6 @@
  */
 package ai.djl.android.core;
 
-import org.junit.Assert;
-import org.junit.Test;
-
-import java.io.IOException;
-
 import ai.djl.ModelException;
 import ai.djl.inference.Predictor;
 import ai.djl.modality.Classifications;
@@ -25,36 +20,63 @@ import ai.djl.modality.cv.ImageFactory;
 import ai.djl.modality.cv.transform.Resize;
 import ai.djl.modality.cv.transform.ToTensor;
 import ai.djl.modality.cv.translator.ImageClassificationTranslator;
+import ai.djl.onnxruntime.zoo.tabular.softmax_regression.IrisClassificationTranslatorFactory;
+import ai.djl.onnxruntime.zoo.tabular.softmax_regression.IrisFlower;
 import ai.djl.repository.zoo.Criteria;
 import ai.djl.repository.zoo.ModelZoo;
 import ai.djl.repository.zoo.ZooModel;
 import ai.djl.translate.TranslateException;
 
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.io.IOException;
+
 /* Integration test to check if model and inference runs fine */
 public class ModelLoadingTest {
 
     @Test
     public void testModelLoading() throws IOException, ModelException, TranslateException {
-
         String modelUrl = "https://resources.djl.ai/demo/pytorch/traced_resnet18.zip";
-        ImageClassificationTranslator.builder()
-            .addTransform(new Resize(224, 224))
-            .addTransform(new ToTensor())
-            .optApplySoftmax(true)
-            .build();
-        Criteria<Image, Classifications> criteria = Criteria.builder()
-            .setTypes(Image.class, Classifications.class)
-            .optModelUrls(modelUrl)
-            .optTranslator(ImageClassificationTranslator.builder()
-                .addTransform(new Resize(224, 224))
-                .addTransform(new ToTensor())
-                .optApplySoftmax(true).build())
-            .build();
-        Image image = ImageFactory.getInstance().fromUrl("https://resources.djl.ai/images/kitten.jpg");
+        ImageClassificationTranslator translator =
+                ImageClassificationTranslator.builder()
+                        .addTransform(new Resize(224, 224))
+                        .addTransform(new ToTensor())
+                        .optApplySoftmax(true)
+                        .build();
+        Criteria<Image, Classifications> criteria =
+                Criteria.builder()
+                        .setTypes(Image.class, Classifications.class)
+                        .optModelUrls(modelUrl)
+                        .optTranslator(translator)
+                        .build();
+
+        ImageFactory factory = ImageFactory.getInstance();
+        Image image = factory.fromUrl("https://resources.djl.ai/images/kitten.jpg");
         try (ZooModel<Image, Classifications> model = ModelZoo.loadModel(criteria);
-             Predictor<Image, Classifications> predictor = model.newPredictor()) {
+                Predictor<Image, Classifications> predictor = model.newPredictor()) {
             Classifications result = predictor.predict(image);
             Assert.assertEquals("n02124075 Egyptian cat", result.best().getClassName());
         }
     }
+
+    @Test
+    public void testONNXRuntimeModelLoading()
+            throws IOException, ModelException, TranslateException {
+        String modelUrl =
+                "https://mlrepo.djl.ai/model/tabular/softmax_regression/ai/djl/onnxruntime/iris_flowers/0.0.1/iris_flowers.zip";
+        Criteria<IrisFlower, Classifications> criteria =
+                Criteria.builder()
+                        .setTypes(IrisFlower.class, Classifications.class)
+                        .optModelUrls(modelUrl)
+                        .optTranslatorFactory(new IrisClassificationTranslatorFactory())
+                        .optEngine("OnnxRuntime") // use OnnxRuntime engine by default
+                        .build();
+
+        ZooModel<IrisFlower, Classifications> model = criteria.loadModel();
+        Predictor<IrisFlower, Classifications> predictor = model.newPredictor();
+        IrisFlower info = new IrisFlower(1.0f, 2.0f, 3.0f, 4.0f);
+        Classifications result = predictor.predict(info);
+        Assert.assertEquals("virginica", result.best().getClassName());
+    }
 }
diff --git a/api/src/test/java/ai/djl/translate/DeferredTranslatorFactoryTest.java b/api/src/test/java/ai/djl/translate/DeferredTranslatorFactoryTest.java
new file mode 100644
index 000000000..faf24818d
--- /dev/null
+++ b/api/src/test/java/ai/djl/translate/DeferredTranslatorFactoryTest.java
@@ -0,0 +1,84 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.translate;
+
+import ai.djl.ModelException;
+import ai.djl.modality.Classifications;
+import ai.djl.modality.cv.Image;
+import ai.djl.nn.Blocks;
+import ai.djl.repository.zoo.Criteria;
+import ai.djl.repository.zoo.ZooModel;
+
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+
+public class DeferredTranslatorFactoryTest {
+
+    @Test
+    public void testDeferredTranslatorFactory() throws ModelException, IOException {
+        DeferredTranslatorFactory factory = new DeferredTranslatorFactory();
+        Assert.assertTrue(factory.getSupportedTypes().isEmpty());
+
+        Path modelPath = Paths.get("src/test/resources/identity");
+        Criteria<Image, Classifications> criteria =
+                Criteria.builder()
+                        .setTypes(Image.class, Classifications.class)
+                        .optModelPath(modelPath)
+                        .optBlock(Blocks.identityBlock())
+                        .optTranslatorFactory(factory)
+                        .build();
+
+        try (ZooModel<Image, Classifications> model = criteria.loadModel()) {
+            Assert.assertNotNull(model);
+        }
+
+        Criteria<Image, Classifications> criteria1 =
+                Criteria.builder()
+                        .setTypes(Image.class, Classifications.class)
+                        .optModelPath(modelPath)
+                        .optBlock(Blocks.identityBlock())
+                        .optTranslatorFactory(factory)
+                        .optArgument("translatorFactory", "")
+                        .build();
+
+        Assert.assertThrows(criteria1::loadModel);
+
+        Criteria<Image, Classifications> criteria2 =
+                Criteria.builder()
+                        .setTypes(Image.class, Classifications.class)
+                        .optModelPath(modelPath)
+                        .optBlock(Blocks.identityBlock())
+                        .optTranslatorFactory(factory)
+                        .optArgument("translatorFactory", "not-exists")
+                        .build();
+
+        Assert.assertThrows(criteria2::loadModel);
+
+        Criteria<Image, Classifications> criteria3 =
+                Criteria.builder()
+                        .setTypes(Image.class, Classifications.class)
+                        .optModelPath(modelPath)
+                        .optBlock(Blocks.identityBlock())
+                        .optTranslatorFactory(factory)
+                        .optArgument(
+                                "translatorFactory",
+                                "ai.djl.modality.cv.translator.StyleTransferTranslatorFactory")
+                        .build();
+
+        Assert.assertThrows(criteria3::loadModel);
+    }
+}
diff --git a/api/src/test/resources/identity/serving.properties b/api/src/test/resources/identity/serving.properties
new file mode 100644
index 000000000..7cda4595a
--- /dev/null
+++ b/api/src/test/resources/identity/serving.properties
@@ -0,0 +1,3 @@
+blockFactory=ai.djl.nn.IdentityBlockFactory
+option.hasParameter=false
+translatorFactory=ai.djl.modality.cv.translator.ImageClassificationTranslatorFactory
diff --git a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizerTest.java b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizerTest.java
index 591df6b22..22cf0782d 100644
--- a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizerTest.java
+++ b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/HuggingFaceTokenizerTest.java
@@ -155,4 +155,77 @@ public class HuggingFaceTokenizerTest {
             }
         }
     }
+
+    @Test
+    public void testTruncationAndPadding() {
+        TestRequirements.notArm();
+
+        try (HuggingFaceTokenizer tokenizer = HuggingFaceTokenizer.newInstance("bert-base-cased")) {
+            List<String> inputs =
+                    Arrays.asList(
+                            "Hello, y'all! How are you?",
+                            "Today is a sunny day. Good weather I'd say",
+                            "I am happy");
+            tokenizer.setTruncationAndPadding(
+                    HuggingFaceTokenizer.TruncationStrategy.LONGEST_FIRST,
+                    HuggingFaceTokenizer.PaddingStrategy.BATCH_LONGEST,
+                    10);
+            List<long[]> expectedIds =
+                    Arrays.asList(
+                            new long[] {101, 8667, 117, 194, 112, 1155, 106, 1731, 1132, 102},
+                            new long[] {101, 3570, 1110, 170, 21162, 1285, 119, 2750, 4250, 102},
+                            new long[] {101, 146, 1821, 2816, 102, 0, 0, 0, 0, 0});
+            List<long[]> expectedAttentionMasks =
+                    Arrays.asList(
+                            new long[] {1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
+                            new long[] {1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
+                            new long[] {1, 1, 1, 1, 1, 0, 0, 0, 0, 0});
+            Encoding[] encodings = tokenizer.batchEncode(inputs);
+            for (int i = 0; i < encodings.length; ++i) {
+                Assert.assertEquals(encodings[i].getIds(), expectedIds.get(i));
+                Assert.assertEquals(encodings[i].getAttentionMask(), expectedAttentionMasks.get(i));
+            }
+
+            tokenizer.setTruncationAndPadding(
+                    HuggingFaceTokenizer.TruncationStrategy.LONGEST_FIRST,
+                    HuggingFaceTokenizer.PaddingStrategy.FIXED,
+                    12);
+            expectedIds =
+                    Arrays.asList(
+                            new long[] {
+                                101, 8667, 117, 194, 112, 1155, 106, 1731, 1132, 1128, 136, 102
+                            },
+                            new long[] {
+                                101, 3570, 1110, 170, 21162, 1285, 119, 2750, 4250, 146, 112, 102
+                            },
+                            new long[] {101, 146, 1821, 2816, 102, 0, 0, 0, 0, 0, 0, 0});
+            expectedAttentionMasks =
+                    Arrays.asList(
+                            new long[] {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
+                            new long[] {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
+                            new long[] {1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0});
+            encodings = tokenizer.batchEncode(inputs);
+            for (int i = 0; i < encodings.length; ++i) {
+                Assert.assertEquals(encodings[i].getIds(), expectedIds.get(i));
+                Assert.assertEquals(encodings[i].getAttentionMask(), expectedAttentionMasks.get(i));
+            }
+
+            tokenizer.setTruncationAndPadding(
+                    HuggingFaceTokenizer.TruncationStrategy.ONLY_FIRST,
+                    HuggingFaceTokenizer.PaddingStrategy.BATCH_LONGEST,
+                    8);
+            Encoding encoding = tokenizer.encode("Hello there my friend", "How are you");
+            long[] expectedId = new long[] {101, 8667, 1175, 102, 1731, 1132, 1128, 102};
+            Assert.assertEquals(encoding.getIds(), expectedId);
+
+            tokenizer.setTruncationAndPadding(
+                    HuggingFaceTokenizer.TruncationStrategy.ONLY_SECOND,
+                    HuggingFaceTokenizer.PaddingStrategy.DO_NOT_PAD,
+                    8);
+
+            encoding = tokenizer.encode("Hello there my friend", "How are you");
+            expectedId = new long[] {101, 8667, 1175, 1139, 1910, 102, 1731, 102};
+            Assert.assertEquals(encoding.getIds(), expectedId);
+        }
+    }
 }
diff --git a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
index 8d2d44561..c4f05d446 100644
--- a/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
+++ b/extensions/tokenizers/src/test/java/ai/djl/huggingface/tokenizers/TranslatorTest.java
@@ -12,9 +12,14 @@
  */
 package ai.djl.huggingface.tokenizers;
 
+import ai.djl.Model;
 import ai.djl.ModelException;
+import ai.djl.huggingface.translator.FillMaskTranslatorFactory;
 import ai.djl.huggingface.translator.QuestionAnsweringTranslatorFactory;
 import ai.djl.inference.Predictor;
+import ai.djl.modality.Classifications;
+import ai.djl.modality.Input;
+import ai.djl.modality.Output;
 import ai.djl.modality.nlp.qa.QAInput;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
@@ -35,6 +40,8 @@ import java.io.IOException;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Map;
 
 public class TranslatorTest {
 
@@ -52,7 +59,6 @@ public class TranslatorTest {
                 "BBC Japan was a general entertainment Channel. "
                         + "Which operated between December 2004 and April 2006. "
                         + "It ceased operations after its Japanese distributor folded.";
-        QAInput input = new QAInput(question, paragraph);
 
         Block block =
                 new LambdaBlock(
@@ -60,8 +66,10 @@ public class TranslatorTest {
                             NDManager manager = a.getManager();
                             long[][] start = new long[1][36];
                             long[][] end = new long[1][36];
-                            start[0][20] = 1;
-                            end[0][21] = 1;
+                            start[0][0] = 2;
+                            start[0][21] = 1;
+                            end[0][0] = 2;
+                            end[0][20] = 1;
                             NDArray arr1 = manager.create(start);
                             NDArray arr2 = manager.create(end);
                             return new NDList(arr1, arr2);
@@ -69,6 +77,7 @@ public class TranslatorTest {
                         "model");
         Path modelDir = Paths.get("build/model");
         Files.createDirectories(modelDir);
+
         Criteria<QAInput, String> criteria =
                 Criteria.builder()
                         .setTypes(QAInput.class, String.class)
@@ -82,8 +91,137 @@ public class TranslatorTest {
 
         try (ZooModel<QAInput, String> model = criteria.loadModel();
                 Predictor<QAInput, String> predictor = model.newPredictor()) {
+            QAInput input = new QAInput(question, paragraph);
             String res = predictor.predict(input);
             Assert.assertEquals(res, "December 2004");
         }
+
+        Criteria<Input, Output> criteria2 =
+                Criteria.builder()
+                        .setTypes(Input.class, Output.class)
+                        .optModelPath(modelDir)
+                        .optBlock(block)
+                        .optEngine("PyTorch")
+                        .optArgument("tokenizer", "bert-base-cased")
+                        .optOption("hasParameter", "false")
+                        .optTranslatorFactory(new QuestionAnsweringTranslatorFactory())
+                        .build();
+
+        try (ZooModel<Input, Output> model = criteria2.loadModel();
+                Predictor<Input, Output> predictor = model.newPredictor()) {
+            Input input = new Input();
+            input.add("question", question);
+            input.add("paragraph", paragraph);
+            Output res = predictor.predict(input);
+            Assert.assertEquals(res.getAsString(0), "December 2004");
+        }
+
+        try (Model model = Model.newInstance("test")) {
+            model.setBlock(block);
+            Map<String, String> options = new HashMap<>();
+            options.put("hasParameter", "false");
+            model.load(modelDir, "test", options);
+
+            QuestionAnsweringTranslatorFactory factory = new QuestionAnsweringTranslatorFactory();
+            Map<String, String> arguments = new HashMap<>();
+
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+
+            arguments.put("tokenizer", "bert-base-cased");
+
+            Assert.assertThrows(
+                    IllegalArgumentException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+        }
+    }
+
+    @Test
+    public void testFillMaskTranslator() throws ModelException, IOException, TranslateException {
+        TestRequirements.notArm();
+
+        String text = "Hello I'm a [MASK] model.";
+
+        Block block =
+                new LambdaBlock(
+                        a -> {
+                            NDManager manager = a.getManager();
+                            float[][] logits = new float[10][4828];
+                            logits[6][4827] = 5;
+                            logits[6][2535] = 4;
+                            logits[6][2047] = 3;
+                            logits[6][3565] = 2;
+                            logits[6][2986] = 1;
+                            NDArray arr = manager.create(logits);
+                            arr = arr.expandDims(0);
+                            return new NDList(arr);
+                        },
+                        "model");
+        Path modelDir = Paths.get("build/model");
+        Files.createDirectories(modelDir);
+
+        Criteria<String, Classifications> criteria =
+                Criteria.builder()
+                        .setTypes(String.class, Classifications.class)
+                        .optModelPath(modelDir)
+                        .optBlock(block)
+                        .optEngine("PyTorch")
+                        .optArgument("tokenizer", "bert-base-uncased")
+                        .optOption("hasParameter", "false")
+                        .optTranslatorFactory(new FillMaskTranslatorFactory())
+                        .build();
+
+        try (ZooModel<String, Classifications> model = criteria.loadModel();
+                Predictor<String, Classifications> predictor = model.newPredictor()) {
+            Classifications res = predictor.predict(text);
+            Assert.assertEquals(res.best().getClassName(), "fashion");
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> predictor.predict("Hello I'm a invalid model."));
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> predictor.predict("I'm a [MASK] [MASK] model."));
+        }
+
+        Criteria<Input, Output> criteria2 =
+                Criteria.builder()
+                        .setTypes(Input.class, Output.class)
+                        .optModelPath(modelDir)
+                        .optBlock(block)
+                        .optEngine("PyTorch")
+                        .optArgument("tokenizer", "bert-base-uncased")
+                        .optOption("hasParameter", "false")
+                        .optTranslatorFactory(new FillMaskTranslatorFactory())
+                        .build();
+
+        try (ZooModel<Input, Output> model = criteria2.loadModel();
+                Predictor<Input, Output> predictor = model.newPredictor()) {
+            Input input = new Input();
+            input.add(text);
+            Output out = predictor.predict(input);
+            Classifications res = (Classifications) out.getData();
+            Assert.assertEquals(res.best().getClassName(), "fashion");
+        }
+
+        try (Model model = Model.newInstance("test")) {
+            model.setBlock(block);
+            Map<String, String> options = new HashMap<>();
+            options.put("hasParameter", "false");
+            model.load(modelDir, "test", options);
+
+            FillMaskTranslatorFactory factory = new FillMaskTranslatorFactory();
+            Map<String, String> arguments = new HashMap<>();
+
+            Assert.assertThrows(
+                    TranslateException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+
+            arguments.put("tokenizer", "bert-base-cased");
+
+            Assert.assertThrows(
+                    IllegalArgumentException.class,
+                    () -> factory.newInstance(String.class, Integer.class, model, arguments));
+        }
     }
 }