Repository: deepjavalibrary/djl
Pull request #1960 (2022-08-25 17:55:18)

[api] Fixes timeseries extension package name
Change-Id: Ib736e38e2a7e7f204d1518ec66d5957277073248

## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/1960?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
> Merging [#1960](https://codecov.io/gh/deepjavalibrary/djl/pull/1960?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (880bc2f) into [master](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) (bb5073f) will **decrease** coverage by `2.10%`.
> The diff coverage is `67.44%`.

```diff
@@             Coverage Diff              @@
##             master    #1960      +/-   ##
============================================
- Coverage     72.08%   69.98%   -2.11%     
- Complexity     5126     5810     +684     
============================================
  Files           473      572      +99     
  Lines         21970    25797    +3827     
  Branches       2351     2779     +428     
============================================
+ Hits          15838    18055    +2217     
- Misses         4925     6376    +1451     
- Partials       1207     1366     +159     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/1960?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [...odality/cv/translator/BigGANTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvckZhY3RvcnkuamF2YQ==) | `33.33% <0.00%> (+8.33%)` | :arrow_up: |
| [...nslator/InstanceSegmentationTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW5zdGFuY2VTZWdtZW50YXRpb25UcmFuc2xhdG9yRmFjdG9yeS5qYXZh) | `14.28% <0.00%> (-3.90%)` | :arrow_down: |
| [.../cv/translator/SemanticSegmentationTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU2VtYW50aWNTZWdtZW50YXRpb25UcmFuc2xhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../cv/translator/StyleTransferTranslatorFactory.java](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvU3R5bGVUcmFuc2ZlclRyYW5zbGF0b3JGYWN0b3J5LmphdmE=) | `40.00% <ø> (ø)` | |
| ... and [473 more](https://codecov.io/gh/deepjavalibrary/djl/pull/1960/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)


Patch:
diff --git a/api/src/main/java/ai/djl/training/evaluator/Coverage.java b/api/src/main/java/ai/djl/training/evaluator/Coverage.java
new file mode 100644
index 000000000..2ee61c09e
--- /dev/null
+++ b/api/src/main/java/ai/djl/training/evaluator/Coverage.java
@@ -0,0 +1,58 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+
+package ai.djl.training.evaluator;
+
+import ai.djl.ndarray.NDArray;
+import ai.djl.ndarray.NDList;
+import ai.djl.util.Pair;
+
+/**
+ * Coverage for a Regression problem: it measures the percent of predictions greater than the actual
+ * target, to determine whether the predictor is over-forecasting or under-forecasting. e.g. 0.50 if
+ * we predict near the median of the distribution.
+ *
+ * <pre>
+ *  def coverage(target, forecast):
+ *     return (np.mean((target &lt; forecast)))
+ * </pre>
+ *
+ * <a href="https://bibinmjose.github.io/2021/03/08/errorblog.html">...</a>
+ */
+public class Coverage extends AbstractAccuracy {
+
+    /**
+     * Creates an evaluator that measures the percent of predictions greater than the actual target.
+     */
+    public Coverage() {
+        this("Coverage", 1);
+    }
+
+    /**
+     * Creates an evaluator that measures the percent of predictions greater than the actual target.
+     *
+     * @param name the name of the evaluator, default is "Coverage"
+     * @param axis the axis along which to count the correct prediction, default is 1
+     */
+    public Coverage(String name, int axis) {
+        super(name, axis);
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    protected Pair<Long, NDArray> accuracyHelper(NDList labels, NDList predictions) {
+        NDArray labl = labels.head();
+        NDArray pred = predictions.head();
+        return new Pair<>(labl.size(), labl.lt(pred));
+    }
+}
diff --git a/api/src/main/java/ai/djl/training/loss/Loss.java b/api/src/main/java/ai/djl/training/loss/Loss.java
index 014f47fef..a661a3e9a 100644
--- a/api/src/main/java/ai/djl/training/loss/Loss.java
+++ b/api/src/main/java/ai/djl/training/loss/Loss.java
@@ -69,7 +69,7 @@ public abstract class Loss extends Evaluator {
     }
 
     /**
-     * Returns a new instance of {@link L1Loss} with given weight and batch axis.
+     * Returns a new instance of {@link L1Loss} with given weight.
      *
      * @param name the name of the loss
      * @param weight the weight to apply on loss value, default 1
@@ -79,6 +79,27 @@ public abstract class Loss extends Evaluator {
         return new L1Loss(name, weight);
     }
 
+    /**
+     * Returns a new instance of {@link QuantileL1Loss} with given quantile.
+     *
+     * @param quantile the quantile position of the data to focus on
+     * @return a new instance of {@link QuantileL1Loss}
+     */
+    public static QuantileL1Loss quantileL1Loss(float quantile) {
+        return new QuantileL1Loss(quantile);
+    }
+
+    /**
+     * Returns a new instance of {@link QuantileL1Loss} with given quantile.
+     *
+     * @param name the name of the loss
+     * @param quantile the quantile position of the data to focus on
+     * @return a new instance of {@link QuantileL1Loss}
+     */
+    public static QuantileL1Loss quantileL1Loss(String name, float quantile) {
+        return new QuantileL1Loss(name, quantile);
+    }
+
     /**
      * Returns a new instance of {@link L2Loss} with default weight and batch axis.
      *
diff --git a/api/src/main/java/ai/djl/training/loss/QuantileL1Loss.java b/api/src/main/java/ai/djl/training/loss/QuantileL1Loss.java
new file mode 100644
index 000000000..d4bb2a987
--- /dev/null
+++ b/api/src/main/java/ai/djl/training/loss/QuantileL1Loss.java
@@ -0,0 +1,71 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.training.loss;
+
+import ai.djl.ndarray.NDArray;
+import ai.djl.ndarray.NDList;
+import ai.djl.ndarray.types.DataType;
+
+/**
+ * {@code QuantileL1Loss} calculates the Weighted Quantile Loss between labels and predictions. It
+ * is useful for regression problems where you wish to estimate a particular quantile. For example,
+ * to target the P90, instantiate {@code new QuantileL1Loss("P90", 0.90)}. Basically, what this loss
+ * function does is to focus on a centain persentile of the data. Eg. q=0.5 is the original default
+ * case of regression, meaning the best-fit line lies in the center. When q=0.9, the best-fit line
+ * will lie above the center; and, if \partial forecast / \partial w are the same, then exactly 0.9
+ * of total data points will lie below the best-fit line.
+ *
+ * <pre>
+ *  def quantile_loss(target, forecast, q):
+ *      return 2 * np.sum(np.abs((forecast - target) * ((target &lt;= forecast) - q)))
+ * </pre>
+ *
+ * <p>Reference: <a href="https://bibinmjose.github.io/2021/03/08/errorblog.html">...</a>
+ */
+public class QuantileL1Loss extends Loss {
+
+    private Number quantile;
+
+    /**
+     * Computes QuantileL1Loss for regression problem.
+     *
+     * @param quantile the quantile position of the data to focus on
+     */
+    public QuantileL1Loss(float quantile) {
+        this("QuantileL1Loss", quantile);
+    }
+
+    /**
+     * Computes QuantileL1Loss for regression problem.
+     *
+     * @param name the name of the loss function, default "QuantileL1Loss"
+     * @param quantile the quantile position of the data to focus on
+     */
+    public QuantileL1Loss(String name, float quantile) {
+        super(name);
+        this.quantile = quantile;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public NDArray evaluate(NDList labels, NDList predictions) {
+        NDArray pred = predictions.singletonOrThrow();
+        NDArray labelReshaped = labels.singletonOrThrow().reshape(pred.getShape());
+        NDArray loss =
+                pred.sub(labelReshaped)
+                        .mul(labelReshaped.lte(pred).toType(DataType.FLOAT32, false).sub(quantile))
+                        .abs()
+                        .mul(2);
+        return loss.mean();
+    }
+}
diff --git a/bom/build.gradle b/bom/build.gradle
index c7119f07e..b37d1b6b8 100644
--- a/bom/build.gradle
+++ b/bom/build.gradle
@@ -45,6 +45,7 @@ dependencies {
         api "ai.djl.tensorflow:tensorflow-model-zoo:${version}"
         api "ai.djl.tensorrt:tensorrt:${version}"
         api "ai.djl.tflite:tflite-engine:${version}"
+        api "ai.djl.timeseries:timeseries:${version}"
     }
 }
 
diff --git a/extensions/timeseries/build.gradle b/extensions/timeseries/build.gradle
index 228874f29..f4aea7010 100644
--- a/extensions/timeseries/build.gradle
+++ b/extensions/timeseries/build.gradle
@@ -1,4 +1,4 @@
-group 'ai.dj.timeseries'
+group 'ai.djl.timeseries'
 
 dependencies {
     api project(":api")


Test patch:
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java b/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
index 597ebb54c..0135f9cfb 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
@@ -19,6 +19,7 @@ import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.training.evaluator.AbstractAccuracy;
 import ai.djl.training.evaluator.Accuracy;
+import ai.djl.training.evaluator.Coverage;
 import ai.djl.training.evaluator.TopKAccuracy;
 
 import org.testng.Assert;
@@ -58,6 +59,25 @@ public class EvaluatorTest {
         }
     }
 
+    @Test
+    public void testCoverage() {
+        try (NDManager manager = NDManager.newBaseManager()) {
+
+            NDArray predictions = manager.create(new float[] {0.3f, 0.7f, 0.0f}, new Shape(3));
+            NDArray labels = manager.create(new float[] {0.5f, 0.5f, 0.5f}, new Shape(3));
+
+            Coverage coverage = new Coverage();
+            coverage.addAccumulator("");
+            coverage.updateAccumulator("", new NDList(labels), new NDList(predictions));
+            float accuracy = coverage.getAccumulator("");
+            float expectedAccuracy = 1.f / 3;
+            Assert.assertEquals(
+                    accuracy,
+                    expectedAccuracy,
+                    "Wrong accuracy, expected: " + expectedAccuracy + ", actual: " + accuracy);
+        }
+    }
+
     @Test
     public void testTopKAccuracy() {
         try (NDManager manager = NDManager.newBaseManager()) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java b/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
index 3989c90e2..1fb74cccc 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
@@ -34,6 +34,19 @@ public class LossTest {
         }
     }
 
+    @Test
+    public void quantileL1LossTest() {
+        try (NDManager manager = NDManager.newBaseManager()) {
+            NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
+            NDArray label = manager.ones(new Shape(5));
+            Assert.assertEquals(
+                    Loss.quantileL1Loss(0.9f)
+                            .evaluate(new NDList(label), new NDList(pred))
+                            .getFloat(),
+                    0.4000001f);
+        }
+    }
+
     @Test
     public void l2LossTest() {
         try (NDManager manager = NDManager.newBaseManager()) {