Repository: deepjavalibrary/djl
Pull request #1412 (2021-12-09 18:27:21)

[sentencepiece] Adds Windows support
## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


frankfliu: @Lundez Thanks for contribution. You should be able to test in your folk (must in master branch).

You can ignore upload to S3 part when running in your folk.

I'm not sure if we need any CMake changes for this PR.
codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
> Merging [#1412](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (65cbf5e) into [master](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (bb5073f) will **decrease** coverage by `1.50%`.
> The diff coverage is `51.08%`.

[![Impacted file tree graph](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/graphs/tree.svg?width=650&height=150&src=pr&token=mDKVNwFXRV&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

```diff
@@             Coverage Diff              @@
##             master    #1412      +/-   ##
============================================
- Coverage     72.08%   70.58%   -1.51%     
- Complexity     5126     5248     +122     
============================================
  Files           473      487      +14     
  Lines         21970    23078    +1108     
  Branches       2351     2518     +167     
============================================
+ Hits          15838    16290     +452     
- Misses         4925     5521     +596     
- Partials       1207     1267      +60     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <ø> (-5.24%)` | :arrow_down: |
| [...odality/cv/translator/BigGANTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `33.33% <0.00%> (+8.33%)` | :arrow_up: |
| [...nslator/InstanceSegmentationTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW5zdGFuY2VTZWdtZW50YXRpb25UcmFuc2xhdG9yRmFjdG9yeS5qYXZh) | `14.28% <0.00%> (-3.90%)` | :arrow_down: |
| [.../modality/cv/translator/YoloTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3JGYWN0b3J5LmphdmE=) | `8.33% <0.00%> (-1.67%)` | :arrow_down: |
| [...i/djl/modality/cv/translator/YoloV5Translator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1Y1VHJhbnNsYXRvci5qYXZh) | `5.69% <0.00%> (ø)` | |
| [...odality/cv/translator/YoloV5TranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1Y1VHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `8.33% <0.00%> (-1.67%)` | :arrow_down: |
| [...modality/nlp/embedding/TrainableWordEmbedding.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L25scC9lbWJlZGRpbmcvVHJhaW5hYmxlV29yZEVtYmVkZGluZy5qYXZh) | `45.00% <0.00%> (-3.34%)` | :arrow_down: |
| [...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL25kYXJyYXkvQnl0ZXNTdXBwbGllci5qYXZh) | `54.54% <0.00%> (-12.13%)` | :arrow_down: |
| [...i/src/main/java/ai/djl/ndarray/NDArrayAdapter.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL25kYXJyYXkvTkRBcnJheUFkYXB0ZXIuamF2YQ==) | `15.23% <0.00%> (-0.83%)` | :arrow_down: |
| ... and [142 more](https://codecov.io/gh/deepjavalibrary/djl/pull/1412/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Last update [e915c1f...65cbf5e](https://codecov.io/gh/deepjavalibrary/djl/pull/1412?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).

Lundez: > @Lundez Thanks for contribution. You should be able to test in your folk (must in master branch).
> 
> You can ignore upload to S3 part when running in your folk.
> 
> I think there are few piece is missing:
> 
> 1. The build.gradle also need to be update: https://github.com/deepjavalibrary/djl/blob/master/extensions/sentencepiece/build.gradle#L64-L68
> 2. We need add a build.cmd file for windows
> 3. I'm not sure if we need any CMake changes for this PR.

Ok, I'll take a look further. This is simply a draft PR for now :) 
Most likely I'll fix it during the weekend!
Lundez: @frankfliu https://github.com/Lundez/djl/actions/runs/1569820490 seems to run :)
Gonna remove AWS comments again
frankfliu: @Lundez 
I made a few fixed on top your PR, but it seems still broken. On Windows, the `.dll` file is not generated.
I believe CMakeList.txt file need to be updated for Windows, see: https://github.com/levyfan/sentencepiece-jni/blob/master/src/main/native/CMakeLists.txt#L50-L56, there might be some other flags needed.
Lundez: > @Lundez 
> I made a few fixed on top your PR, but it seems still broken. On Windows, the `.dll` file is not generated.
> I believe CMakeList.txt file need to be updated for Windows, see: https://github.com/levyfan/sentencepiece-jni/blob/master/src/main/native/CMakeLists.txt#L50-L56, there might be some other flags needed.

I'll take a look tomorrow. 👍
Lundez: Don't have windows myself, so hard to test this. I'll give it some more time tomorrow, been very busy today.
Lundez: @frankfliu it seems to be created based on the logs: https://pipelines.actions.githubusercontent.com/vaIqj4dCv5ckseiiqJ6l5KKSICSYkSf0qDCf97KwQ9WmgmyDyY/_apis/pipelines/1/runs/5/signedlogcontent/4?urlExpires=2021-12-14T16%3A24%3A23.3573049Z&urlSigningMethod=HMACV1&urlSignature=nD0kwdqTjE8qtFLbjNQgkDQw8DXgtDIcZKlY%2FmFM7uU%3D

> 2021-12-14T16:19:15.1459755Z   sentencepiece_native.vcxproj -> D:\a\djl\djl\extensions\sentencepiece\build\Release\sentencepiece_native.dll

> 2021-12-14T16:20:38.1959663Z     [Test worker] INFO ai.djl.sentencepiece.jni.LibUtils - Extracting /native/lib/win-x86_64/sentencepiece_native.dll to cache ...

though it fails on 
> Gradle suite > Gradle test > ai.djl.sentencepiece.SpTextEmbeddingTest > testEmbedding FAILED 2021-12-14T16:20:38.1970138Z     java.lang.IllegalStateException at SpTextEmbeddingTest.java:39

I'm not sure how I debug in GitHub Actions. And as I don't have Windows it's hard to run locally.. Ideas? Or is the DLL still missing you think?
frankfliu: @Lundez No worry, we will look into this.

@stu1130 can you review CMake changes for windows?
Lundez: @frankfliu got more logging on (`--info`)

> Gradle suite > Gradle test > ai.djl.sentencepiece.SpVocabularyTest > testTokenIdConversion FAILED
    java.lang.IllegalStateException: SentencePiece library not found: /native/lib/win-x86_64/sentencepiece_native.dll
        at ai.djl.sentencepiece.jni.LibUtils.copyJniLibraryFromClasspath(LibUtils.java:66)

Patch:
diff --git a/.github/workflows/continuous.yml b/.github/workflows/continuous.yml
index 1f4b381b3..71d79e73e 100644
--- a/.github/workflows/continuous.yml
+++ b/.github/workflows/continuous.yml
@@ -73,6 +73,8 @@ jobs:
           key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
           restore-keys: |
             ${{ runner.os }}-gradle-
+      - name: Compile Sentencepiece JNI
+        run: ./gradlew :extensions:sentencepiece:compileJNI
       - name: Compile PyTorch JNI
         run: ./gradlew :engines:pytorch:pytorch-native:compileJNI
       - name: Compile PaddlePaddle JNI
diff --git a/.github/workflows/native_s3_sentencepiece.yml b/.github/workflows/native_s3_sentencepiece.yml
index 27557ca24..962b3ad58 100644
--- a/.github/workflows/native_s3_sentencepiece.yml
+++ b/.github/workflows/native_s3_sentencepiece.yml
@@ -71,3 +71,34 @@ jobs:
           SENTENCEPIECE_VERSION="$(cat gradle.properties | awk -F '=' '/sentencepiece_version/ {print $2}')"
           aws s3 sync extensions/sentencepiece/jnilib s3://djl-ai/publish/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/
           aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
+
+  build-sentencepiece-jni-windows:
+    runs-on: windows-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up JDK 1.8
+        uses: actions/setup-java@v1
+        with:
+          java-version: 1.8
+      - uses: actions/cache@v2
+        with:
+          path: ~/.gradle/caches
+          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
+          restore-keys: |
+            ${{ runner.os }}-gradle-
+      - name: Release JNI prep
+        run: |
+          ./gradlew :extensions:sentencepiece:compileJNI
+          ./gradlew -Pjni :extensions:sentencepiece:test
+      - name: Configure AWS Credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: us-east-2
+      - name: Copy files to S3 with the AWS CLI
+        shell: bash
+        run: |
+          SENTENCEPIECE_VERSION="$(cat gradle.properties | awk -F '=' '/sentencepiece_version/ {print $2}')"
+          aws s3 sync extensions/sentencepiece/jnilib s3://djl-ai/publish/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/
+          aws cloudfront create-invalidation --distribution-id E371VB8JQ6NRVY --paths "/sentencepiece-${SENTENCEPIECE_VERSION}/jnilib/*"
diff --git a/extensions/sentencepiece/CMakeLists.txt b/extensions/sentencepiece/CMakeLists.txt
index 54566a936..7aea0fd21 100644
--- a/extensions/sentencepiece/CMakeLists.txt
+++ b/extensions/sentencepiece/CMakeLists.txt
@@ -9,6 +9,14 @@ set(SPM_ENABLE_TCMALLOC OFF CACHE BOOL "Build sentencepiece static library")
 # disable shared library as we only link static library
 set(SPM_ENABLE_SHARED OFF CACHE BOOL "Build sentencepiece static library")
 
+if (MSVC)
+  string(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_DEBUG          ${CMAKE_CXX_FLAGS_DEBUG})
+  string(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_MINSIZEREL     ${CMAKE_CXX_FLAGS_MINSIZEREL})
+  string(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_RELEASE        ${CMAKE_CXX_FLAGS_RELEASE})
+  string(REPLACE "/MD" "/MT" CMAKE_CXX_FLAGS_RELWITHDEBINFO ${CMAKE_CXX_FLAGS_RELWITHDEBINFO})
+  add_definitions("/wd4267 /wd4244 /wd4305 /Zc:strictStrings /utf-8")
+endif()
+
 find_package(JNI REQUIRED)
 find_path(UTILS_INCLUDE_DIR NAMES djl/utils.h PATHS ${PROJECT_SOURCE_DIR}/../../api/src/main/native REQUIRED)
 
diff --git a/extensions/sentencepiece/build.cmd b/extensions/sentencepiece/build.cmd
new file mode 100644
index 000000000..1253de17c
--- /dev/null
+++ b/extensions/sentencepiece/build.cmd
@@ -0,0 +1,25 @@
+set VERSION="%1"
+set FILEPATH="sentencepiece"
+
+if exist %FILEPATH% (
+    echo Found %FILEPATH%
+) else (
+    echo Couldn't find %FILEPATH%, downloading it ...
+    echo Cloning google/sentencepiece.git
+    git clone https://github.com/google/sentencepiece.git -b %VERSION%
+    echo Finished cloning sentencepiece
+)
+
+if exist build rd /q /s build
+md build\classes
+cd build
+
+javac -sourcepath ../src/main/java/ ../src/main/java/ai/djl/sentencepiece/jni/SentencePieceLibrary.java -h include -d classes
+cmake ..
+cmake --build . --config Release
+
+:: for nightly ci
+md jnilib\win-x86_64
+copy Release\sentencepiece_native.dll jnilib\win-x86_64
+
+cd ..
diff --git a/extensions/sentencepiece/build.gradle b/extensions/sentencepiece/build.gradle
index e46229873..7dd054cff 100644
--- a/extensions/sentencepiece/build.gradle
+++ b/extensions/sentencepiece/build.gradle
@@ -19,6 +19,7 @@ processResources {
     doLast {
         def url = "https://publish.djl.ai/sentencepiece-${sentencepiece_version}/jnilib/${djl_version}"
         def files = [
+                "win-x86_64"  : "sentencepiece_native.dll",
                 "linux-x86_64": "libsentencepiece_native.so",
                 "osx-x86_64"  : "libsentencepiece_native.dylib"
         ]
@@ -61,22 +62,26 @@ apply from: file("${rootProject.projectDir}/tools/gradle/cpp-formatter.gradle")
 
 task compileJNI {
     doFirst {
-        if (System.properties['os.name'].toLowerCase(Locale.ROOT).contains("mac")
-                || System.properties['os.name'].toLowerCase(Locale.ROOT).contains("linux")) {
+        if (System.properties['os.name'].toLowerCase(Locale.ROOT).contains("win")) {
             exec {
-                commandLine 'bash', 'build.sh'
+                commandLine "${project.projectDir}/build.cmd", "v${sentencepiece_version}"
             }
-
-            // for ci to upload to S3
-            def ciDir = "${project.projectDir}/jnilib/${djl_version}/"
-            copy {
-                from "${project.buildDir}/jnilib"
-                into ciDir
+        } else if (System.properties['os.name'].toLowerCase(Locale.ROOT).contains("mac")
+                || System.properties['os.name'].toLowerCase(Locale.ROOT).contains("linux")) {
+            exec {
+                commandLine "bash", "build.sh", "v${sentencepiece_version}"
             }
-            delete System.getProperty("user.home") + "/.djl.ai/sentencepiece"
         } else {
             throw new IllegalStateException("Unknown Architecture " + System.properties['os.name'])
         }
+
+        // for ci to upload to S3
+        def ciDir = "${project.projectDir}/jnilib/${djl_version}/"
+        copy {
+            from "${project.buildDir}/jnilib"
+            into ciDir
+        }
+        delete System.getProperty("user.home") + "/.djl.ai/sentencepiece"
     }
 }
 
diff --git a/extensions/sentencepiece/build.sh b/extensions/sentencepiece/build.sh
index e82dd9198..c1ddb9852 100755
--- a/extensions/sentencepiece/build.sh
+++ b/extensions/sentencepiece/build.sh
@@ -10,7 +10,7 @@ elif [[ -n $(command -v sysctl) ]]; then
 fi
 PLATFORM=$(uname | tr '[:upper:]' '[:lower:]')
 
-VERSION=v0.1.95
+VERSION=$1
 
 pushd $WORK_DIR
 if [ ! -d "sentencepiece" ];
diff --git a/extensions/sentencepiece/src/main/java/ai/djl/sentencepiece/jni/LibUtils.java b/extensions/sentencepiece/src/main/java/ai/djl/sentencepiece/jni/LibUtils.java
index 7a51d7471..09821b9cd 100644
--- a/extensions/sentencepiece/src/main/java/ai/djl/sentencepiece/jni/LibUtils.java
+++ b/extensions/sentencepiece/src/main/java/ai/djl/sentencepiece/jni/LibUtils.java
@@ -43,10 +43,6 @@ public final class LibUtils {
     private LibUtils() {}
 
     public static void loadLibrary() {
-        if (System.getProperty("os.name").startsWith("Win")) {
-            throw new UnsupportedOperationException("Windows is not supported.");
-        }
-
         String libName = copyJniLibraryFromClasspath();
         logger.debug("Loading sentencepiece library from: {}", libName);
         System.load(libName); // NOPMD
diff --git a/extensions/sentencepiece/src/main/native/ai_djl_sentencepiece_jni_SentencePieceLibrary.cc b/extensions/sentencepiece/src/main/native/ai_djl_sentencepiece_jni_SentencePieceLibrary.cc
index cf6d1c73c..de0d94261 100644
--- a/extensions/sentencepiece/src/main/native/ai_djl_sentencepiece_jni_SentencePieceLibrary.cc
+++ b/extensions/sentencepiece/src/main/native/ai_djl_sentencepiece_jni_SentencePieceLibrary.cc
@@ -96,7 +96,7 @@ JNIEXPORT jstring JNICALL Java_ai_djl_sentencepiece_jni_SentencePieceLibrary_idT
   return env->NewStringUTF(processor_ptr->IdToPiece(jid).c_str());
 }
 
-JNIEXPORT int JNICALL Java_ai_djl_sentencepiece_jni_SentencePieceLibrary_pieceToId(
+JNIEXPORT jint JNICALL Java_ai_djl_sentencepiece_jni_SentencePieceLibrary_pieceToId(
     JNIEnv* env, jobject jthis, jlong jhandle, jstring jpiece) {
   auto* processor_ptr = reinterpret_cast<sentencepiece::SentencePieceProcessor*>(jhandle);
   return processor_ptr->PieceToId(djl::utils::jni::GetStringFromJString(env, jpiece));


Test patch:
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
index 811e1e216..18c4fb8c3 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTextEmbeddingTest.java
@@ -12,7 +12,6 @@
  */
 package ai.djl.sentencepiece;
 
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.util.DownloadUtils;
 import java.io.IOException;
 import java.nio.file.Files;
@@ -36,8 +35,6 @@ public class SpTextEmbeddingTest {
 
     @Test
     public void testEmbedding() throws IOException {
-        TestRequirements.notWindows();
-
         Path modelPath = Paths.get("build/test/models/sententpiece_test_model.model");
         try (SpTokenizer tokenizer = new SpTokenizer(modelPath)) {
             SpTextEmbedding embedding = SpTextEmbedding.from(tokenizer);
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
index e14360390..155290e37 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpTokenizerTest.java
@@ -39,8 +39,6 @@ public class SpTokenizerTest {
 
     @Test
     public void testLoadFromBytes() throws IOException {
-        TestRequirements.notWindows();
-
         Path modelPath = Paths.get("build/test/models/sententpiece_test_model.model");
         byte[] bytes = Files.readAllBytes(modelPath);
         try (SpTokenizer tokenizer = new SpTokenizer(bytes)) {
diff --git a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
index ec3eaddd6..fbc4d4970 100644
--- a/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
+++ b/extensions/sentencepiece/src/test/java/ai/djl/sentencepiece/SpVocabularyTest.java
@@ -13,7 +13,6 @@
 
 package ai.djl.sentencepiece;
 
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.util.DownloadUtils;
 import java.io.IOException;
 import java.nio.file.Files;
@@ -37,8 +36,6 @@ public class SpVocabularyTest {
 
     @Test
     public void testTokenIdConversion() throws IOException {
-        TestRequirements.notWindows();
-
         Path modelPath = Paths.get("build/test/models");
         String prefix = "sententpiece_test_model";
         SpTokenizer tokenizer = new SpTokenizer(modelPath, prefix);