Repository: deepjavalibrary/djl
Pull request #2325 (2023-01-16 22:30:59)

[sentencepiece] Adds aarch64 support for sentencepiece
## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
Base: **72.08**% // Head: **74.41**% // Increases project coverage by **`+2.32%`** :tada:
> Coverage data is based on head [(`74676ad`)](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) compared to base [(`bb5073f`)](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> Patch coverage: 75.10% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##             master    #2325      +/-   ##
============================================
+ Coverage     72.08%   74.41%   +2.32%     
- Complexity     5126     6714    +1588     
============================================
  Files           473      659     +186     
  Lines         21970    29153    +7183     
  Branches       2351     3016     +665     
============================================
+ Hits          15838    21694    +5856     
- Misses         4925     5990    +1065     
- Partials       1207     1469     +262     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [.../modality/cv/translator/ImageFeatureExtractor.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW1hZ2VGZWF0dXJlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../ai/djl/modality/cv/translator/YoloTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3IuamF2YQ==) | `27.77% <0.00%> (+18.95%)` | :arrow_up: |
| [...ain/java/ai/djl/modality/cv/util/NDImageUtils.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3V0aWwvTkRJbWFnZVV0aWxzLmphdmE=) | `67.10% <0.00%> (+7.89%)` | :arrow_up: |
| [api/src/main/java/ai/djl/modality/nlp/Decoder.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L25scC9EZWNvZGVyLmphdmE=) | `63.63% <ø> (ø)` | |
| ... and [604 more](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2325?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/.github/workflows/native_s3_sentencepiece.yml b/.github/workflows/native_s3_sentencepiece.yml
index a08bdbb79..c3c03f148 100644
--- a/.github/workflows/native_s3_sentencepiece.yml
+++ b/.github/workflows/native_s3_sentencepiece.yml
@@ -5,6 +5,7 @@ on:
 
 jobs:
   build-sentencepiece-jni-osx:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
     runs-on: macos-latest
     steps:
       - uses: actions/checkout@v3
@@ -36,6 +37,7 @@ jobs:
           aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
 
   build-sentencepiece-jni-linux:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
     runs-on: ubuntu-latest
     container: amazonlinux:2
     steps:
@@ -75,6 +77,7 @@ jobs:
           aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
 
   build-sentencepiece-jni-windows:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
     runs-on: windows-latest
     steps:
       - uses: actions/checkout@v3
@@ -105,3 +108,115 @@ jobs:
           SENTENCEPIECE_VERSION="$(cat gradle.properties | awk -F '=' '/sentencepiece_version/ {print $2}')"
           aws s3 sync extensions/sentencepiece/jnilib s3://djl-ai/publish/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/
           aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
+
+  build-sentencepiece-jni-arm64-osx:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
+    runs-on: [ self-hosted, ARM64, macOS ]
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up JDK 11
+        uses: actions/setup-java@v3
+        with:
+          java-version: 11
+          distribution: corretto
+          architecture: aarch64
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+      - uses: actions/cache@v3
+        with:
+          path: ~/.gradle/caches
+          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
+          restore-keys: |
+            ${{ runner.os }}-gradle-
+      - name: Release JNI prep
+        run: |
+          ./gradlew :extensions:sentencepiece:compileJNI
+          ./gradlew -Pjni :extensions:sentencepiece:test
+      - name: Configure AWS Credentials
+        uses: aws-actions/configure-aws-credentials@v1-node16
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: us-east-2
+      - name: Copy files to S3 with the AWS CLI
+        run: |
+          SENTENCEPIECE_VERSION="$(cat gradle.properties | awk -F '=' '/sentencepiece_version/ {print $2}')"
+          aws s3 sync extensions/sentencepiece/jnilib s3://djl-ai/publish/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/
+          aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
+
+  build-sentencepiece-jni-aarch64:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
+    runs-on: [ self-hosted, aarch64 ]
+    timeout-minutes: 30
+    needs: create-aarch64-runner
+    container: centos:centos7
+    steps:
+      - name: Install Environment
+        run: |
+          yum -y update
+          yum -y install centos-release-scl-rh epel-release
+          yum -y install devtoolset-7 git patch cmake3 libstdc++-static
+          ln -s /usr/bin/cmake3 /usr/bin/cmake
+          curl https://sh.rustup.rs -sSf | sh -s -- -y
+          pip3 install awscli --upgrade
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+      - uses: actions/checkout@v3
+      - name: Set up JDK 11
+        uses: actions/setup-java@v3
+        with:
+          java-version: 11
+          distribution: corretto
+          architecture: aarch64
+      - uses: actions/cache@v3
+        with:
+          path: ~/.gradle/caches
+          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
+          restore-keys: |
+            ${{ runner.os }}-gradle-
+      - name: Release JNI prep
+        run: |
+          source "$HOME/.cargo/env"
+          export PATH=$PATH:/opt/rh/devtoolset-7/root/usr/bin
+          ./gradlew :extensions:sentencepiece:compileJNI
+          ./gradlew -Pjni :extensions:sentencepiece:test
+      - name: Configure AWS Credentials
+        uses: aws-actions/configure-aws-credentials@v1-node16
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: us-east-2
+      - name: Copy files to S3 with the AWS CLI
+        run: |
+          SENTENCEPIECE_VERSION="$(cat gradle.properties | awk -F '=' '/sentencepiece_version/ {print $2}')"
+          aws s3 sync extensions/sentencepiece/jnilib s3://djl-ai/publish/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/
+          aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
+
+  create-aarch64-runner:
+    if: github.repository == 'deepjavalibrary/djl'
+    runs-on: [ self-hosted, scheduler ]
+    steps:
+      - name: Create new Graviton instance
+        id: create_aarch64
+        run: |
+          cd /home/ubuntu/djl_benchmark_script/scripts
+          token=$( curl -X POST -H "Authorization: token ${{ secrets.ACTION_RUNNER_PERSONAL_TOKEN }}" \
+          https://api.github.com/repos/deepjavalibrary/djl/actions/runners/registration-token \
+          --fail \
+          | jq '.token' | tr -d '"' )
+          ./start_instance.sh action_graviton $token djl
+    outputs:
+      aarch64_instance_id: ${{ steps.create_aarch64.outputs.action_graviton_instance_id }}
+
+  stop-runners:
+    if: ${{ github.repository == 'deepjavalibrary/djl' && always() }}
+    runs-on: [ self-hosted, scheduler ]
+    needs: [ create-aarch64-runner, build-sentencepiece-jni-aarch64 ]
+    steps:
+      - name: Stop all instances
+        run: |
+          cd /home/ubuntu/djl_benchmark_script/scripts
+          instance_id=${{ needs.create-aarch64-runner.outputs.aarch64_instance_id }}
+          ./stop_instance.sh $instance_id
diff --git a/extensions/sentencepiece/build.gradle b/extensions/sentencepiece/build.gradle
index 1b9e2b313..43fa5fef7 100644
--- a/extensions/sentencepiece/build.gradle
+++ b/extensions/sentencepiece/build.gradle
@@ -64,8 +64,9 @@ task compileJNI {
             }
         } else if (System.properties['os.name'].toLowerCase(Locale.ROOT).contains("mac")
                 || System.properties['os.name'].toLowerCase(Locale.ROOT).contains("linux")) {
+            def arch = System.properties["os.arch"] == "amd64" ? "x86_64" : System.properties["os.arch"]
             exec {
-                commandLine "bash", "build.sh", "v${sentencepiece_version}"
+                commandLine "bash", "build.sh", "v${sentencepiece_version}", arch
             }
         } else {
             throw new IllegalStateException("Unknown Architecture " + System.properties['os.name'])
diff --git a/extensions/sentencepiece/build.sh b/extensions/sentencepiece/build.sh
index c1ddb9852..08536b2ef 100755
--- a/extensions/sentencepiece/build.sh
+++ b/extensions/sentencepiece/build.sh
@@ -11,6 +11,7 @@ fi
 PLATFORM=$(uname | tr '[:upper:]' '[:lower:]')
 
 VERSION=$1
+ARCH=$2
 
 pushd $WORK_DIR
 if [ ! -d "sentencepiece" ];
@@ -34,9 +35,9 @@ popd
 
 # for nightly ci
 if [[ $PLATFORM == 'darwin' ]]; then
-  mkdir -p build/jnilib/osx-x86_64
-  cp -f build/libsentencepiece_native.dylib build/jnilib/osx-x86_64/
+  mkdir -p build/jnilib/osx-$ARCH
+  cp -f build/libsentencepiece_native.dylib build/jnilib/osx-$ARCH/
 elif [[ $PLATFORM == 'linux' ]]; then
-  mkdir -p build/jnilib/linux-x86_64
-  cp -f build/libsentencepiece_native.so build/jnilib/linux-x86_64/
+  mkdir -p build/jnilib/linux-$ARCH
+  cp -f build/libsentencepiece_native.so build/jnilib/linux-$ARCH/
 fi
diff --git a/extensions/tokenizers/build.gradle b/extensions/tokenizers/build.gradle
index 3a2f4313d..429bdd137 100644
--- a/extensions/tokenizers/build.gradle
+++ b/extensions/tokenizers/build.gradle
@@ -1,3 +1,5 @@
+import java.util.zip.GZIPInputStream
+
 group "ai.djl.huggingface"
 
 dependencies {
@@ -44,6 +46,27 @@ processResources {
         // write properties
         def propFile = file("${project.buildDir}/classes/java/main/native/lib/tokenizers.properties")
         propFile.text = "version=${tokenizers_version}-${version}\n"
+
+        url = "https://mlrepo.djl.ai/model/nlp"
+        def tasks = [
+                "fill_mask",
+                "question_answer",
+                "text_classification",
+                "text_embedding",
+                "token_classification",
+        ]
+        def prefix = "${project.buildDir}/classes/java/main/nlp"
+        for (String task : tasks) {
+            def file = new File("${prefix}/${task}/ai.djl.huggingface.pytorch.json")
+            if (file.exists()) {
+                project.logger.lifecycle("model zoo metadata alrady exists: ${task}")
+            } else {
+                project.logger.lifecycle("Downloading model zoo metadata: ${task}")
+                file.getParentFile().mkdirs()
+                def downloadPath = new URL("${url}/${task}/ai/djl/huggingface/pytorch/models.json.gz")
+                downloadPath.withInputStream { i -> file.withOutputStream { it << new GZIPInputStream(i) } }
+            }
+        }
     }
 }
 
diff --git a/extensions/tokenizers/src/main/java/ai/djl/huggingface/zoo/HfModelZoo.java b/extensions/tokenizers/src/main/java/ai/djl/huggingface/zoo/HfModelZoo.java
index 88cb137b2..78346217f 100644
--- a/extensions/tokenizers/src/main/java/ai/djl/huggingface/zoo/HfModelZoo.java
+++ b/extensions/tokenizers/src/main/java/ai/djl/huggingface/zoo/HfModelZoo.java
@@ -19,6 +19,7 @@ import ai.djl.repository.Repository;
 import ai.djl.repository.Version;
 import ai.djl.repository.VersionRange;
 import ai.djl.repository.zoo.ModelZoo;
+import ai.djl.util.ClassLoaderUtils;
 import ai.djl.util.JsonUtils;
 import ai.djl.util.Utils;
 
@@ -28,10 +29,10 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import java.io.IOException;
+import java.io.InputStream;
 import java.io.Reader;
 import java.io.Writer;
 import java.lang.reflect.Type;
-import java.net.URL;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.time.Duration;
@@ -117,13 +118,30 @@ public class HfModelZoo extends ModelZoo {
 
             String url = REPO + path + "models.json.gz";
             Path tmp = Files.createTempFile(dir, "models", ".tmp");
-            try (GZIPInputStream gis = new GZIPInputStream(new URL(url).openStream())) {
+            try (GZIPInputStream gis = new GZIPInputStream(Utils.openUrl(url))) {
                 String json = Utils.toString(gis);
                 try (Writer writer = Files.newBufferedWriter(tmp)) {
                     writer.write(json);
                 }
                 Utils.moveQuietly(tmp, file);
                 return JsonUtils.GSON.fromJson(json, type);
+            } catch (IOException e) {
+                logger.warn("Failed to download Huggingface model zoo index: {}", app);
+                if (Files.exists(file)) {
+                    try (Reader reader = Files.newBufferedReader(file)) {
+                        return JsonUtils.GSON.fromJson(reader, type);
+                    }
+                }
+
+                String resource = app.getPath() + "/" + GROUP_ID + ".json";
+                try (InputStream is = ClassLoaderUtils.getResourceAsStream(resource)) {
+                    String json = Utils.toString(is);
+                    try (Writer writer = Files.newBufferedWriter(tmp)) {
+                        writer.write(json);
+                    }
+                    Utils.moveQuietly(tmp, file);
+                    return JsonUtils.GSON.fromJson(json, type);
+                }
             } finally {
                 Utils.deleteQuietly(tmp);
             }
diff --git a/extensions/tokenizers/src/main/python/huggingface_converter.py b/extensions/tokenizers/src/main/python/huggingface_converter.py
index 5b4749d28..f3b85c241 100644
--- a/extensions/tokenizers/src/main/python/huggingface_converter.py
+++ b/extensions/tokenizers/src/main/python/huggingface_converter.py
@@ -112,7 +112,7 @@ class HuggingfaceConverter:
             logging.info(f"Saving torchscript model: {model_name}.pt ...")
             model_file = os.path.join(temp_dir, f"{model_name}.pt")
             script_module.save(model_file)
-        except RuntimeError as e:
+        except (RuntimeError, ValueError) as e:
             logging.warning(f"Failed to trace model: {model_id}.")
             logging.warning(e, exc_info=True)
             return None


Test patch:
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
index 8d816f203..360c32388 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
@@ -12,7 +12,6 @@
  */
 package ai.djl.sentencepiece;
 
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.util.DownloadUtils;
 
 import org.testng.Assert;
@@ -26,10 +25,9 @@ import java.nio.file.Paths;
 import java.util.Collections;
 
 public class SpTextEmbeddingTest {
+
     @BeforeTest
     public void downloadModel() throws IOException {
-        TestRequirements.notArm();
-
         Path modelFile = Paths.get("build/test/models/sententpiece_test_model.model");
         if (Files.notExists(modelFile)) {
             DownloadUtils.download(
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
index 4821609ed..92a1b0ab1 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
@@ -32,8 +32,6 @@ public class SpTokenizerTest {
 
     @BeforeTest
     public void downloadModel() throws IOException {
-        TestRequirements.notArm();
-
         Path modelFile = Paths.get("build/test/sp_model/sp_model.model");
         if (Files.notExists(modelFile)) {
             DownloadUtils.download(
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
index f34279539..a0e565d54 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
@@ -13,7 +13,6 @@
 
 package ai.djl.sentencepiece;
 
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.util.DownloadUtils;
 
 import org.testng.Assert;
@@ -29,8 +28,6 @@ public class SpVocabularyTest {
 
     @BeforeTest
     public void downloadModel() throws IOException {
-        TestRequirements.notArm();
-
         Path modelFile = Paths.get("build/test/models/sententpiece_test_model.model");
         if (Files.notExists(modelFile)) {
             DownloadUtils.download(
diff --git a/extensions/tokenizers/src/test/java/ai/djl/huggingface/zoo/ModelZooTest.java b/extensions/tokenizers/src/test/java/ai/djl/huggingface/zoo/ModelZooTest.java
index 89e19014d..e585a219f 100644
--- a/extensions/tokenizers/src/test/java/ai/djl/huggingface/zoo/ModelZooTest.java
+++ b/extensions/tokenizers/src/test/java/ai/djl/huggingface/zoo/ModelZooTest.java
@@ -31,7 +31,9 @@ import java.io.IOException;
 import java.io.Writer;
 import java.nio.file.Files;
 import java.nio.file.Path;
+import java.nio.file.Paths;
 import java.util.Map;
+import java.util.Set;
 import java.util.concurrent.ConcurrentHashMap;
 
 public class ModelZooTest {
@@ -62,6 +64,7 @@ public class ModelZooTest {
     public void testFutureVersion() throws IOException {
         System.setProperty("DJL_CACHE_DIR", "build/cache");
         try {
+            Utils.deleteQuietly(Paths.get("build/cache"));
             Map<String, Map<String, Object>> map = new ConcurrentHashMap<>();
             Map<String, Object> model = new ConcurrentHashMap<>();
             model.put("result", "failed");
@@ -96,4 +99,25 @@ public class ModelZooTest {
             System.clearProperty("DJL_CACHE_DIR");
         }
     }
+
+    @Test
+    public void testOffLine() throws IOException {
+        System.setProperty("DJL_CACHE_DIR", "build/cache");
+        System.setProperty("offline", "true");
+        try {
+            Utils.deleteQuietly(Paths.get("build/cache"));
+            // static variables cannot not be initialized properly if directly use new HfModelZoo()
+            ModelZoo.getModelZoo("ai.djl.huggingface.pytorch");
+
+            ModelZoo zoo = new HfModelZoo();
+            Assert.assertTrue(zoo.getModelLoaders().size() > 0);
+
+            Set<String> engines = zoo.getSupportedEngines();
+            Assert.assertEquals(engines.size(), 1);
+            Assert.assertEquals(engines.iterator().next(), "PyTorch");
+        } finally {
+            System.clearProperty("DJL_CACHE_DIR");
+            System.clearProperty("offline");
+        }
+    }
 }