Repository: deepjavalibrary/djl
Pull request #2129 (2022-11-03 23:49:02)

LightGBM inference result matches input type
This makes the LightGBM inference return the same as the input type (either float32 or float64) rather than always returning float64 per the API.

In addition, it adds a bit nicer handling for creation from a Buffer in that it will also accept either a FloatBuffer or a DoubleBuffer.


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2129?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
Base: **72.08**% // Head: **71.43**% // Decreases project coverage by **`-0.65%`** :warning:
> Coverage data is based on head [(`21db500`)](https://codecov.io/gh/deepjavalibrary/djl/pull/2129?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) compared to base [(`bb5073f`)](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> Patch coverage: 71.84% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##             master    #2129      +/-   ##
============================================
- Coverage     72.08%   71.43%   -0.66%     
- Complexity     5126     6313    +1187     
============================================
  Files           473      624     +151     
  Lines         21970    27871    +5901     
  Branches       2351     3009     +658     
============================================
+ Hits          15838    19909    +4071     
- Misses         4925     6502    +1577     
- Partials       1207     1460     +253     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/2129?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [.../modality/cv/translator/ImageFeatureExtractor.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW1hZ2VGZWF0dXJlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../ai/djl/modality/cv/translator/YoloTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3IuamF2YQ==) | `27.77% <0.00%> (+18.95%)` | :arrow_up: |
| [...modality/cv/translator/wrapper/FileTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3Ivd3JhcHBlci9GaWxlVHJhbnNsYXRvci5qYXZh) | `44.44% <ø> (ø)` | |
| [...y/cv/translator/wrapper/InputStreamTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3Ivd3JhcHBlci9JbnB1dFN0cmVhbVRyYW5zbGF0b3IuamF2YQ==) | `44.44% <ø> (ø)` | |
| ... and [558 more](https://codecov.io/gh/deepjavalibrary/djl/pull/2129/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2129?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDManager.java b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDManager.java
index d12cec31d..6d6757d02 100644
--- a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDManager.java
+++ b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDManager.java
@@ -24,6 +24,8 @@ import ai.djl.ndarray.types.Shape;
 import java.nio.Buffer;
 import java.nio.ByteBuffer;
 import java.nio.ByteOrder;
+import java.nio.DoubleBuffer;
+import java.nio.FloatBuffer;
 import java.nio.file.Path;
 
 /** {@code LgbmNDManager} is the LightGBM implementation of {@link NDManager}. */
@@ -74,11 +76,23 @@ public class LgbmNDManager extends BaseNDManager {
         if (data instanceof ByteBuffer) {
             // output only NDArray
             return new LgbmNDArray(this, alternativeManager, (ByteBuffer) data, shape, dataType);
+        } else if (data instanceof FloatBuffer && dataType == DataType.FLOAT32) {
+            ByteBuffer bb = ByteBuffer.allocateDirect(data.capacity() * 4);
+            bb.asFloatBuffer().put((FloatBuffer) data);
+            bb.rewind();
+            return new LgbmNDArray(this, alternativeManager, bb, shape, dataType);
+        } else if (data instanceof DoubleBuffer && dataType == DataType.FLOAT64) {
+            ByteBuffer bb = ByteBuffer.allocateDirect(data.capacity() * 8);
+            bb.asDoubleBuffer().put((DoubleBuffer) data);
+            bb.rewind();
+            return new LgbmNDArray(this, alternativeManager, bb, shape, dataType);
         }
         if (alternativeManager != null) {
             return alternativeManager.create(data, shape, dataType);
         }
-        throw new UnsupportedOperationException("LgbmNDArray only supports float32.");
+        throw new UnsupportedOperationException(
+                "LgbmNDArray only supports float32 and float64. Please pass either a ByteBuffer, a"
+                        + " FloatBuffer with Float32, or a DoubleBuffer with Float64.");
     }
 
     /** {@inheritDoc} */
diff --git a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmSymbolBlock.java b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmSymbolBlock.java
index 1b4b0a29b..0bb92645a 100644
--- a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmSymbolBlock.java
+++ b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmSymbolBlock.java
@@ -15,12 +15,12 @@ package ai.djl.ml.lightgbm;
 import ai.djl.ml.lightgbm.jni.JniUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
-import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.nn.AbstractSymbolBlock;
 import ai.djl.nn.ParameterList;
 import ai.djl.nn.SymbolBlock;
 import ai.djl.training.ParameterStore;
+import ai.djl.util.Pair;
 import ai.djl.util.PairList;
 
 import com.microsoft.ml.lightgbm.SWIGTYPE_p_p_void;
@@ -64,13 +64,14 @@ public class LgbmSymbolBlock extends AbstractSymbolBlock implements AutoCloseabl
         NDArray array = inputs.singletonOrThrow();
         try (LgbmNDManager sub = (LgbmNDManager) manager.newSubManager()) {
             LgbmNDArray lgbmNDArray = sub.from(array);
-            // TODO: return DirectBuffer from JNI to avoid copy
-            double[] result = JniUtils.inference(handle.get(), iterations, lgbmNDArray);
-            ByteBuffer buf = manager.allocateDirect(result.length * 8);
-            buf.asDoubleBuffer().put(result);
-            buf.rewind();
+            Pair<Integer, ByteBuffer> result =
+                    JniUtils.inference(handle.get(), iterations, lgbmNDArray);
 
-            NDArray ret = manager.create(buf, new Shape(result.length), DataType.FLOAT64);
+            NDArray ret =
+                    manager.create(
+                            result.getValue(),
+                            new Shape(result.getKey()),
+                            lgbmNDArray.getDataType());
             ret.attach(array.getManager());
             return new NDList(ret);
         }
diff --git a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/jni/JniUtils.java b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/jni/JniUtils.java
index 73ece41b0..aebd49106 100644
--- a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/jni/JniUtils.java
+++ b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/jni/JniUtils.java
@@ -18,6 +18,8 @@ import ai.djl.ml.lightgbm.LgbmNDArray;
 import ai.djl.ml.lightgbm.LgbmNDManager;
 import ai.djl.ml.lightgbm.LgbmSymbolBlock;
 import ai.djl.ndarray.NDArray;
+import ai.djl.ndarray.types.DataType;
+import ai.djl.util.Pair;
 
 import com.microsoft.ml.lightgbm.SWIGTYPE_p_double;
 import com.microsoft.ml.lightgbm.SWIGTYPE_p_int;
@@ -26,6 +28,10 @@ import com.microsoft.ml.lightgbm.SWIGTYPE_p_p_void;
 import com.microsoft.ml.lightgbm.lightgbmlib;
 import com.microsoft.ml.lightgbm.lightgbmlibJNI;
 
+import java.nio.ByteBuffer;
+import java.nio.DoubleBuffer;
+import java.nio.FloatBuffer;
+
 /** DJL class that has access to LightGBM JNI. */
 @SuppressWarnings("MissingJavadocMethod")
 public final class JniUtils {
@@ -53,7 +59,8 @@ public final class JniUtils {
         checkCall(result);
     }
 
-    public static double[] inference(SWIGTYPE_p_p_void model, int iterations, NDArray a) {
+    public static Pair<Integer, ByteBuffer> inference(
+            SWIGTYPE_p_p_void model, int iterations, NDArray a) {
         if (a instanceof LgbmDataset) {
             LgbmDataset dataset = (LgbmDataset) a;
             switch (dataset.getSrcType()) {
@@ -72,7 +79,8 @@ public final class JniUtils {
         throw new IllegalArgumentException("LightGBM inference must be called with a LgbmNDArray");
     }
 
-    public static double[] inferenceMat(SWIGTYPE_p_p_void model, int iterations, LgbmNDArray a) {
+    public static Pair<Integer, ByteBuffer> inferenceMat(
+            SWIGTYPE_p_p_void model, int iterations, LgbmNDArray a) {
         SWIGTYPE_p_long_long outLength = lightgbmlib.new_int64_tp();
         SWIGTYPE_p_double outBuffer = null;
         try {
@@ -92,12 +100,29 @@ public final class JniUtils {
                             outLength,
                             outBuffer);
             checkCall(result);
-            long length = lightgbmlib.int64_tp_value(outLength);
-            double[] values = new double[(int) length];
-            for (int i = 0; i < length; i++) {
-                values[i] = lightgbmlib.doubleArray_getitem(outBuffer, i);
+            int length = Math.toIntExact(lightgbmlib.int64_tp_value(outLength));
+            if (a.getDataType() == DataType.FLOAT32) {
+                ByteBuffer bb = ByteBuffer.allocateDirect(length * 4);
+                FloatBuffer wrapped = bb.asFloatBuffer();
+                for (int i = 0; i < length; i++) {
+                    wrapped.put((float) lightgbmlib.doubleArray_getitem(outBuffer, i));
+                }
+                bb.rewind();
+                return new Pair<>(length, bb);
+            } else if (a.getDataType() == DataType.FLOAT64) {
+                ByteBuffer bb = ByteBuffer.allocateDirect(length * 8);
+                DoubleBuffer wrapped = bb.asDoubleBuffer();
+                for (int i = 0; i < length; i++) {
+                    wrapped.put(lightgbmlib.doubleArray_getitem(outBuffer, i));
+                }
+                bb.rewind();
+                return new Pair<>(length, bb);
+            } else {
+                throw new IllegalArgumentException(
+                        "Unexpected data type for LightGBM inference. Expected Float32 or Float64,"
+                                + " but found "
+                                + a.getDataType());
             }
-            return values;
         } catch (EngineException e) {
             throw new EngineException("Failed to run inference using LightGBM native engine", e);
         } finally {
diff --git a/engines/paddlepaddle/paddlepaddle-native/src/main/native/ai_djl_paddlepaddle_jni_PaddleLibrary_inference.cc b/engines/paddlepaddle/paddlepaddle-native/src/main/native/ai_djl_paddlepaddle_jni_PaddleLibrary_inference.cc
index e1b78545d..7c91a9d69 100644
--- a/engines/paddlepaddle/paddlepaddle-native/src/main/native/ai_djl_paddlepaddle_jni_PaddleLibrary_inference.cc
+++ b/engines/paddlepaddle/paddlepaddle-native/src/main/native/ai_djl_paddlepaddle_jni_PaddleLibrary_inference.cc
@@ -81,16 +81,16 @@ JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_analysisConfig
   config_ptr->pass_builder()->DeletePass(djl::utils::jni::GetStringFromJString(env, jpass));
 }
 
-JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_analysisConfigEnableONNXRuntime
-        (JNIEnv* env, jobject jthis, jlong jhandle) {
-    auto* config_ptr = reinterpret_cast<paddle::AnalysisConfig*>(jhandle);
-    config_ptr->EnableONNXRuntime();
+JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_analysisConfigEnableONNXRuntime(
+    JNIEnv* env, jobject jthis, jlong jhandle) {
+  auto* config_ptr = reinterpret_cast<paddle::AnalysisConfig*>(jhandle);
+  config_ptr->EnableONNXRuntime();
 }
 
-JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_analysisConfigEnableORTOptimization
-        (JNIEnv* env, jobject jthis, jlong jhandle) {
-    auto* config_ptr = reinterpret_cast<paddle::AnalysisConfig*>(jhandle);
-    config_ptr->EnableORTOptimization();
+JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_analysisConfigEnableORTOptimization(
+    JNIEnv* env, jobject jthis, jlong jhandle) {
+  auto* config_ptr = reinterpret_cast<paddle::AnalysisConfig*>(jhandle);
+  config_ptr->EnableORTOptimization();
 }
 
 JNIEXPORT void JNICALL Java_ai_djl_paddlepaddle_jni_PaddleLibrary_deleteAnalysisConfig(
diff --git a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_tensor.cc b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_tensor.cc
index 2b89fb98f..f81a66e44 100644
--- a/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_tensor.cc
+++ b/engines/pytorch/pytorch-native/src/main/native/ai_djl_pytorch_jni_PyTorchLibrary_tensor.cc
@@ -334,7 +334,7 @@ JNIEXPORT jboolean JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchRequiresG
     JNIEnv* env, jobject jthis, jlong jhandle) {
   API_BEGIN()
   const auto* tensor_ptr = reinterpret_cast<torch::Tensor*>(jhandle);
-  return tensor_ptr->requires_grad();
+  return tensor_ptr->requires_grad() && (tensor_ptr->is_leaf() || tensor_ptr->retains_grad());
   API_END_RETURN()
 }
 


Test patch:
diff --git a/engines/ml/lightgbm/src/test/java/ai/djl/ml/lightgbm/LgbmModelTest.java b/engines/ml/lightgbm/src/test/java/ai/djl/ml/lightgbm/LgbmModelTest.java
index 19c3cc1d2..da45a1c03 100644
--- a/engines/ml/lightgbm/src/test/java/ai/djl/ml/lightgbm/LgbmModelTest.java
+++ b/engines/ml/lightgbm/src/test/java/ai/djl/ml/lightgbm/LgbmModelTest.java
@@ -55,7 +55,7 @@ public class LgbmModelTest {
             try (NDManager manager = NDManager.newBaseManager()) {
                 NDArray array = manager.ones(new Shape(10, 4));
                 NDList output = predictor.predict(new NDList(array));
-                Assert.assertEquals(output.singletonOrThrow().getDataType(), DataType.FLOAT64);
+                Assert.assertEquals(output.singletonOrThrow().getDataType(), DataType.FLOAT32);
                 Assert.assertEquals(output.singletonOrThrow().getShape().size(), 10);
             }
         }
diff --git a/integration/src/test/java/ai/djl/integration/IntegrationTests.java b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
index 3ee337282..93cd6c12d 100644
--- a/integration/src/test/java/ai/djl/integration/IntegrationTests.java
+++ b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
@@ -46,6 +46,7 @@ public class IntegrationTests {
         for (String engine : engines) {
             System.setProperty("ai.djl.default_engine", engine);
             logger.info("Testing engine: {} ...", engine);
+            logger.info("=============== {} ===============", engine);
             Assert.assertTrue(new IntegrationTest(IntegrationTest.class).runTests(args));
             // currently each engine will reserve a certain amount of memory and hold it until
             // process terminate so running 3 different engines sequentially without