Repository: deepjavalibrary/djl
Pull request #2725 (2023-07-26 23:53:50)

Fix AmazonReviews
The amazon reviews had a dependency on another s3 bucket that was removed, breaking the CI. This fixes it by uploading it to the mlrepo s3 bucket.

codecov-commenter: ## [Codecov](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
Patch coverage: **`54.72%`** and project coverage change: **`+0.01%`** :tada:
> Comparison is base [(`bb5073f`)](https://app.codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) 72.08% compared to head [(`991d335`)](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) 72.10%.
> Report is 859 commits behind head on master.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please [install the Github App Integration](https://github.com/apps/codecov) for your organization. [Read more](https://about.codecov.io/blog/codecov-is-updating-its-github-integration/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##             master    #2725      +/-   ##
============================================
+ Coverage     72.08%   72.10%   +0.01%     
- Complexity     5126     7027    +1901     
============================================
  Files           473      698     +225     
  Lines         21970    31270    +9300     
  Branches       2351     3225     +874     
============================================
+ Hits          15838    22546    +6708     
- Misses         4925     7190    +2265     
- Partials       1207     1534     +327     
```


| [Files Changed](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...djl/modality/cv/transform/RandomFlipLeftRight.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zZm9ybS9SYW5kb21GbGlwTGVmdFJpZ2h0LmphdmE=) | `25.00% <0.00%> (-25.00%)` | :arrow_down: |
| [...djl/modality/cv/transform/RandomFlipTopBottom.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zZm9ybS9SYW5kb21GbGlwVG9wQm90dG9tLmphdmE=) | `25.00% <0.00%> (-25.00%)` | :arrow_down: |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [.../modality/cv/translator/ImageFeatureExtractor.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW1hZ2VGZWF0dXJlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../ai/djl/modality/cv/translator/YoloTranslator.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3IuamF2YQ==) | `27.77% <0.00%> (+18.95%)` | :arrow_up: |
| [...ain/java/ai/djl/modality/cv/util/NDImageUtils.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3V0aWwvTkRJbWFnZVV0aWxzLmphdmE=) | `67.10% <0.00%> (+7.89%)` | :arrow_up: |
| [api/src/main/java/ai/djl/modality/nlp/Decoder.java](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L25scC9EZWNvZGVyLmphdmE=) | `63.63% <ø> (ø)` | |
| ... and [226 more](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

... and [368 files with indirect coverage changes](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/deepjavalibrary/djl/pull/2725?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
index e4bfcc76b..45e217746 100644
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -27,7 +27,7 @@ jobs:
       - name: Install CN fonts
         run: sudo apt-get update && sudo apt-get install fonts-arphic-uming
       - name: install Python Dependencies
-        run: pip3 install nbconvert mkdocs mkdocs-exclude mknotebooks mkdocs-material jupyter Pygments Markdown
+        run: pip3 install nbconvert mkdocs==1.4.3 mkdocs-exclude mknotebooks mkdocs-material jupyter Pygments Markdown
       - name: Install IJava kernel
         run: |
           git clone https://github.com/frankfliu/IJava.git
diff --git a/api/src/main/java/ai/djl/repository/zoo/BaseModelLoader.java b/api/src/main/java/ai/djl/repository/zoo/BaseModelLoader.java
index cb5e68921..89de4c254 100644
--- a/api/src/main/java/ai/djl/repository/zoo/BaseModelLoader.java
+++ b/api/src/main/java/ai/djl/repository/zoo/BaseModelLoader.java
@@ -30,6 +30,9 @@ import ai.djl.util.Pair;
 import ai.djl.util.Progress;
 import ai.djl.util.Utils;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import java.io.IOException;
 import java.io.Reader;
 import java.lang.reflect.Type;
@@ -43,6 +46,8 @@ import java.util.stream.Collectors;
 /** Shared code for the {@link ModelLoader} implementations. */
 public class BaseModelLoader implements ModelLoader {
 
+    private static final Logger logger = LoggerFactory.getLogger(BaseModelLoader.class);
+
     protected MRL mrl;
     protected TranslatorFactory defaultFactory;
 
@@ -238,6 +243,9 @@ public class BaseModelLoader implements ModelLoader {
         if (factoryClass != null) {
             ClassLoader cl = ClassLoaderUtils.getContextClassLoader();
             factory = ClassLoaderUtils.initClass(cl, TranslatorFactory.class, factoryClass);
+            if (factory == null) {
+                logger.warn("Failed to load translatorFactory: {}", factoryClass);
+            }
         }
         return factory;
     }
diff --git a/api/src/main/java/ai/djl/util/ClassLoaderUtils.java b/api/src/main/java/ai/djl/util/ClassLoaderUtils.java
index d5661a8c7..47141fe8d 100644
--- a/api/src/main/java/ai/djl/util/ClassLoaderUtils.java
+++ b/api/src/main/java/ai/djl/util/ClassLoaderUtils.java
@@ -82,7 +82,11 @@ public final class ClassLoaderUtils {
                             (PrivilegedAction<ClassLoader>)
                                     () -> new URLClassLoader(urls, contextCl));
             if (className != null && !className.isEmpty()) {
-                return initClass(cl, type, className);
+                T impl = initClass(cl, type, className);
+                if (impl == null) {
+                    logger.warn("Failed to load class: {}", className);
+                }
+                return impl;
             }
 
             T implemented = scanDirectory(cl, type, classesDir);
diff --git a/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainAmazonReviewRanking.java b/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainAmazonReviewRanking.java
index 2122b719a..d87626a1c 100644
--- a/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainAmazonReviewRanking.java
+++ b/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainAmazonReviewRanking.java
@@ -125,7 +125,7 @@ public final class TrainAmazonReviewRanking {
     private static CsvDataset getDataset(
             Arguments arguments, BertFullTokenizer tokenizer, int maxLength) {
         String amazonReview =
-                "https://s3.amazonaws.com/amazon-reviews-pds/tsv/amazon_reviews_us_Digital_Software_v1_00.tsv.gz";
+                "https://mlrepo.djl.ai/dataset/nlp/ai/djl/basicdataset/amazon_reviews/1.0/amazon_reviews_us_Digital_Software_v1_00.tsv.gz";
         float paddingToken = tokenizer.getVocabulary().getIndex("[PAD]");
         return CsvDataset.builder()
                 .optCsvUrl(amazonReview)
diff --git a/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb b/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb
index 411fbbf0a..2edbc6c19 100644
--- a/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb
+++ b/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb
@@ -159,7 +159,7 @@
    "source": [
     "CsvDataset getDataset(int batchSize, BertFullTokenizer tokenizer, int maxLength, int limit) {\n",
     "    String amazonReview =\n",
-    "            \"https://s3.amazonaws.com/amazon-reviews-pds/tsv/amazon_reviews_us_Digital_Software_v1_00.tsv.gz\";\n",
+    "            \"https://mlrepo.djl.ai/dataset/nlp/ai/djl/basicdataset/amazon_reviews/1.0/amazon_reviews_us_Digital_Software_v1_00.tsv.gz\";\n",
     "    float paddingToken = tokenizer.getVocabulary().getIndex(\"[PAD]\");\n",
     "    return CsvDataset.builder()\n",
     "            .optCsvUrl(amazonReview) // load from Url\n",


Test patch:
diff --git a/basicdataset/src/test/resources/mlrepo/dataset/nlp/ai/djl/basicdataset/amazon_reviews/metadata.json b/basicdataset/src/test/resources/mlrepo/dataset/nlp/ai/djl/basicdataset/amazon_reviews/metadata.json
index 16f4fcd22..373b4fe9e 100644
--- a/basicdataset/src/test/resources/mlrepo/dataset/nlp/ai/djl/basicdataset/amazon_reviews/metadata.json
+++ b/basicdataset/src/test/resources/mlrepo/dataset/nlp/ai/djl/basicdataset/amazon_reviews/metadata.json
@@ -17,24 +17,9 @@
       },
       "files": {
         "amazon_reviews": {
-          "uri": "https://s3.amazonaws.com/amazon-reviews-pds/tsv/amazon_reviews_us_Digital_Software_v1_00.tsv.gz",
-          "sha1Hash": "b8390100b92579ed814eede4112514417e339902",
-          "size": 18997559
-        }
-      }
-    },
-    {
-      "version": "1.0",
-      "snapshot": false,
-      "name": "amazon_reviews_us_Software",
-      "properties": {
-        "dataset": "us_Software"
-      },
-      "files": {
-        "amazon_reviews": {
-          "uri": "https://s3.amazonaws.com/amazon-reviews-pds/tsv/amazon_reviews_us_Software_v1_00.tsv.gz",
-          "sha1Hash": "e48346dd356698ce680e385e3ecf07501de695b8",
-          "size": 94010685
+          "uri": "1.0/amazon_reviews_us_Digital_Software_v1_00.tsv.gz",
+          "sha1Hash": "098fb62c5731161dd1e10298a5d11636253609a1",
+          "size": 18997604
         }
       }
     }