Repository: deepjavalibrary/djl
Pull request #2279 (2023-01-02 22:08:11)

[api] Improve hybrid engine operators
- [integration] Increase test coverage for NDArrays.java
- [api] Improve hybrid engine operators

## Description ##

Brief description of what this PR is about

- If this change is a backward incompatible change, why must this change be made?
- Interesting edge cases to note here


codecov-commenter: # [Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2279?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) Report
Base: **72.08**% // Head: **72.44**% // Increases project coverage by **`+0.36%`** :tada:
> Coverage data is based on head [(`6b0c768`)](https://codecov.io/gh/deepjavalibrary/djl/pull/2279?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) compared to base [(`bb5073f`)](https://codecov.io/gh/deepjavalibrary/djl/commit/bb5073f7ffc33f1883621fc1783dbb01e70b418f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).
> Patch coverage: 75.38% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##             master    #2279      +/-   ##
============================================
+ Coverage     72.08%   72.44%   +0.36%     
- Complexity     5126     6567    +1441     
============================================
  Files           473      631     +158     
  Lines         21970    28250    +6280     
  Branches       2351     3002     +651     
============================================
+ Hits          15838    20467    +4629     
- Misses         4925     6320    +1395     
- Partials       1207     1463     +256     
```


| [Impacted Files](https://codecov.io/gh/deepjavalibrary/djl/pull/2279?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | Coverage Δ | |
|---|---|---|
| [api/src/main/java/ai/djl/modality/cv/Image.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L0ltYWdlLmphdmE=) | `69.23% <ø> (-4.11%)` | :arrow_down: |
| [...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L011bHRpQm94UHJpb3IuamF2YQ==) | `76.00% <ø> (ø)` | |
| [...rc/main/java/ai/djl/modality/cv/output/Joints.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9Kb2ludHMuamF2YQ==) | `71.42% <ø> (ø)` | |
| [.../main/java/ai/djl/modality/cv/output/Landmark.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9MYW5kbWFyay5qYXZh) | `100.00% <ø> (ø)` | |
| [...main/java/ai/djl/modality/cv/output/Rectangle.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L291dHB1dC9SZWN0YW5nbGUuamF2YQ==) | `72.41% <0.00%> (ø)` | |
| [...i/djl/modality/cv/translator/BigGANTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvQmlnR0FOVHJhbnNsYXRvci5qYXZh) | `21.42% <0.00%> (-5.24%)` | :arrow_down: |
| [.../modality/cv/translator/ImageFeatureExtractor.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvSW1hZ2VGZWF0dXJlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../ai/djl/modality/cv/translator/YoloTranslator.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3RyYW5zbGF0b3IvWW9sb1RyYW5zbGF0b3IuamF2YQ==) | `27.77% <0.00%> (+18.95%)` | :arrow_up: |
| [...ain/java/ai/djl/modality/cv/util/NDImageUtils.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L2N2L3V0aWwvTkRJbWFnZVV0aWxzLmphdmE=) | `59.21% <0.00%> (ø)` | |
| [api/src/main/java/ai/djl/modality/nlp/Decoder.java](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary#diff-YXBpL3NyYy9tYWluL2phdmEvYWkvZGpsL21vZGFsaXR5L25scC9EZWNvZGVyLmphdmE=) | `63.63% <ø> (ø)` | |
| ... and [569 more](https://codecov.io/gh/deepjavalibrary/djl/pull/2279/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/deepjavalibrary/djl/pull/2279?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=deepjavalibrary).


Patch:
diff --git a/api/src/main/java/ai/djl/ndarray/NDArrayAdapter.java b/api/src/main/java/ai/djl/ndarray/NDArrayAdapter.java
index 5ed86277c..2c70258f9 100644
--- a/api/src/main/java/ai/djl/ndarray/NDArrayAdapter.java
+++ b/api/src/main/java/ai/djl/ndarray/NDArrayAdapter.java
@@ -121,12 +121,18 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public DataType getDataType() {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         return dataType;
     }
 
     /** {@inheritDoc} */
     @Override
     public Shape getShape() {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         return shape;
     }
 
@@ -188,13 +194,13 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray gather(NDArray index, int axis) {
-        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
+        return getAlternativeArray().gather(alternativeManager.from(index), axis);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray gatherNd(NDArray index) {
-        throw new UnsupportedOperationException("Not implemented yet.");
+        return getAlternativeArray().gatherNd(alternativeManager.from(index));
     }
 
     /** {@inheritDoc} */
@@ -267,13 +273,13 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray booleanMask(NDArray index, int axis) {
-        return getAlternativeArray().booleanMask(index, axis);
+        return getAlternativeArray().booleanMask(alternativeManager.from(index), axis);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray sequenceMask(NDArray sequenceLength, float value) {
-        return getAlternativeArray().sequenceMask(sequenceLength, value);
+        return getAlternativeArray().sequenceMask(alternativeManager.from(sequenceLength), value);
     }
 
     /** {@inheritDoc} */
@@ -291,7 +297,10 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public boolean contentEquals(NDArray other) {
-        return Arrays.equals(toByteArray(), other.toByteArray());
+        if (other instanceof NDArrayAdapter) {
+            return Arrays.equals(toByteArray(), other.toByteArray());
+        }
+        return other.contentEquals(this);
     }
 
     /** {@inheritDoc} */
@@ -303,7 +312,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray eq(NDArray other) {
-        return getAlternativeArray().eq(other);
+        return getAlternativeArray().eq(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -315,7 +324,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray neq(NDArray other) {
-        return getAlternativeArray().neq(other);
+        return getAlternativeArray().neq(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -327,7 +336,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray gt(NDArray other) {
-        return getAlternativeArray().gt(other);
+        return getAlternativeArray().gt(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -339,7 +348,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray gte(NDArray other) {
-        return getAlternativeArray().gte(other);
+        return getAlternativeArray().gte(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -351,7 +360,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray lt(NDArray other) {
-        return getAlternativeArray().lt(other);
+        return getAlternativeArray().lt(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -363,7 +372,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray lte(NDArray other) {
-        return getAlternativeArray().lte(other);
+        return getAlternativeArray().lte(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -375,7 +384,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray add(NDArray other) {
-        return getAlternativeArray().add(other);
+        return getAlternativeArray().add(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -387,7 +396,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray sub(NDArray other) {
-        return getAlternativeArray().sub(other);
+        return getAlternativeArray().sub(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -399,7 +408,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray mul(NDArray other) {
-        return getAlternativeArray().mul(other);
+        return getAlternativeArray().mul(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -411,7 +420,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray div(NDArray other) {
-        return getAlternativeArray().div(other);
+        return getAlternativeArray().div(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -423,7 +432,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray mod(NDArray other) {
-        return getAlternativeArray().mod(other);
+        return getAlternativeArray().mod(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -435,79 +444,79 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray pow(NDArray other) {
-        return getAlternativeArray().pow(other);
+        return getAlternativeArray().pow(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray addi(Number n) {
-        return getAlternativeArray().addi(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray addi(NDArray other) {
-        return getAlternativeArray().addi(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray subi(Number n) {
-        return getAlternativeArray().subi(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray subi(NDArray other) {
-        return getAlternativeArray().subi(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray muli(Number n) {
-        return getAlternativeArray().muli(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray muli(NDArray other) {
-        return getAlternativeArray().muli(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray divi(Number n) {
-        return getAlternativeArray().divi(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray divi(NDArray other) {
-        return getAlternativeArray().divi(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray modi(Number n) {
-        return getAlternativeArray().modi(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray modi(NDArray other) {
-        return getAlternativeArray().modi(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray powi(Number n) {
-        return getAlternativeArray().powi(n);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray powi(NDArray other) {
-        return getAlternativeArray().powi(other);
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
@@ -519,7 +528,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray signi() {
-        return getAlternativeArray().signi();
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
@@ -531,7 +540,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray maximum(NDArray other) {
-        return getAlternativeArray().maximum(other);
+        return getAlternativeArray().maximum(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -543,7 +552,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray minimum(NDArray other) {
-        return getAlternativeArray().minimum(other);
+        return getAlternativeArray().minimum(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -555,7 +564,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray negi() {
-        return getAlternativeArray().negi();
+        throw new UnsupportedOperationException(UNSUPPORTED_MSG);
     }
 
     /** {@inheritDoc} */
@@ -765,7 +774,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray cumProd(int axis, DataType dataType) {
-        return getAlternativeArray().cumProd(axis);
+        return getAlternativeArray().cumProd(axis, dataType);
     }
 
     /** {@inheritDoc} */
@@ -873,19 +882,19 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray logicalAnd(NDArray other) {
-        return getAlternativeArray().logicalAnd(other);
+        return getAlternativeArray().logicalAnd(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray logicalOr(NDArray other) {
-        return getAlternativeArray().logicalOr(other);
+        return getAlternativeArray().logicalOr(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
     @Override
     public NDArray logicalXor(NDArray other) {
-        return getAlternativeArray().logicalXor(other);
+        return getAlternativeArray().logicalXor(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -1113,7 +1122,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray inverse() {
-        throw new UnsupportedOperationException("Not implemented yet.");
+        return getAlternativeArray().inverse();
     }
 
     /** {@inheritDoc} */
@@ -1137,7 +1146,7 @@ public abstract class NDArrayAdapter implements NDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray batchDot(NDArray other) {
-        return getAlternativeArray().batchDot(other);
+        return getAlternativeArray().batchDot(alternativeManager.from(other));
     }
 
     /** {@inheritDoc} */
@@ -1175,6 +1184,21 @@ public abstract class NDArrayAdapter implements NDArray {
         }
     }
 
+    /** {@inheritDoc} */
+    @Override
+    public boolean equals(Object obj) {
+        if (obj instanceof NDArray) {
+            return contentEquals((NDArray) obj);
+        }
+        return false;
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public int hashCode() {
+        return 0;
+    }
+
     /** {@inheritDoc} */
     @Override
     public String toString() {
diff --git a/api/src/main/java/ai/djl/nn/AbstractBaseBlock.java b/api/src/main/java/ai/djl/nn/AbstractBaseBlock.java
index cb0d6ebe5..1a19d1185 100644
--- a/api/src/main/java/ai/djl/nn/AbstractBaseBlock.java
+++ b/api/src/main/java/ai/djl/nn/AbstractBaseBlock.java
@@ -313,7 +313,7 @@ public abstract class AbstractBaseBlock implements Block {
      * Overwrite this to load additional metadata with the parameter values.
      *
      * <p>If you overwrite {@link AbstractBlock#saveMetadata(DataOutputStream)} or need to provide
-     * backward compatibility to older binary formats, you prabably need to overwrite this. This
+     * backward compatibility to older binary formats, you probably need to overwrite this. This
      * default implementation checks if the version number fits, if not it throws an {@link
      * MalformedModelException}. After that it restores the input shapes.
      *
diff --git a/api/src/main/java/ai/djl/nn/AbstractBlock.java b/api/src/main/java/ai/djl/nn/AbstractBlock.java
index 5bd415085..761780888 100644
--- a/api/src/main/java/ai/djl/nn/AbstractBlock.java
+++ b/api/src/main/java/ai/djl/nn/AbstractBlock.java
@@ -131,7 +131,7 @@ public abstract class AbstractBlock extends AbstractBaseBlock {
     }
 
     /**
-     * Adds a parameter to this block. If parameters are added with this method, intialization of
+     * Adds a parameter to this block. If parameters are added with this method, initialization of
      * the parameter works out of the box
      *
      * @param <P> the specific parameter subclass
diff --git a/api/src/main/java/ai/djl/nn/Block.java b/api/src/main/java/ai/djl/nn/Block.java
index 0adf2e457..b81d79529 100644
--- a/api/src/main/java/ai/djl/nn/Block.java
+++ b/api/src/main/java/ai/djl/nn/Block.java
@@ -61,7 +61,7 @@ import java.util.function.Predicate;
  * <p>We provide helpers for creating two common structures of blocks. For blocks that call children
  * in a chain, use {@link SequentialBlock}. If a blocks calls all of the children in parallel and
  * then combines their results, use {@link ParallelBlock}. For blocks that do not fit these
- * strcutures, you should directly extend the {@link AbstractBlock} class.
+ * structures, you should directly extend the {@link AbstractBlock} class.
  *
  * <p>A block does not necessarily have to have children and parameters. For example, {@link
  * SequentialBlock}, and {@link ParallelBlock} don't have any parameters, but do have child blocks.
@@ -95,9 +95,9 @@ import java.util.function.Predicate;
  * ai.djl.ndarray.NDArray} that are initialized according to the {@link Initializer} set. At this
  * stage, the block is expecting a specific type of input, and is ready to be trained.
  *
- * <p>Training is when we starting feeding the training data as input to the block, get the output,
- * and try to update parameters to learn. For more information about training, please refer the
- * javadoc at {@link ai.djl.training.Trainer}. At the end of training, a block represents a
+ * <p>Training is when we are starting feeding the training data as input to the block, get the
+ * output, and try to update parameters to learn. For more information about training, please refer
+ * the javadoc at {@link ai.djl.training.Trainer}. At the end of training, a block represents a
  * fully-trained model.
  *
  * @see <a
diff --git a/api/src/main/java/ai/djl/training/dataset/RandomAccessDataset.java b/api/src/main/java/ai/djl/training/dataset/RandomAccessDataset.java
index 0d3e6a45c..4326add48 100644
--- a/api/src/main/java/ai/djl/training/dataset/RandomAccessDataset.java
+++ b/api/src/main/java/ai/djl/training/dataset/RandomAccessDataset.java
@@ -101,7 +101,7 @@ public abstract class RandomAccessDataset implements Dataset {
     /**
      * Fetches an iterator that can iterate through the {@link Dataset} with a custom sampler.
      *
-     * @param manager the dataset to iterate through
+     * @param manager the manager to create the arrays
      * @param sampler the sampler to use to iterate through the dataset
      * @return an {@link Iterable} of {@link Batch} that contains batches of data from the dataset
      * @throws IOException for various exceptions depending on the dataset
@@ -116,7 +116,7 @@ public abstract class RandomAccessDataset implements Dataset {
      * Fetches an iterator that can iterate through the {@link Dataset} with a custom sampler
      * multi-threaded.
      *
-     * @param manager the dataset to iterate through
+     * @param manager the manager to create the arrays
      * @param sampler the sampler to use to iterate through the dataset
      * @param executorService the executorService to multi-thread with
      * @return an {@link Iterable} of {@link Batch} that contains batches of data from the dataset
@@ -294,25 +294,25 @@ public abstract class RandomAccessDataset implements Dataset {
      * <p>Each Number[] is a flattened dataset record and the Number[][] is the array of all
      * records.
      *
+     * @param manager the manager to create the arrays
      * @return the dataset contents as a Java array
      * @throws IOException for various exceptions depending on the dataset
      * @throws TranslateException if there is an error while processing input
      */
-    public Pair<Number[][], Number[][]> toArray() throws IOException, TranslateException {
-        try (NDManager manager = NDManager.newBaseManager()) {
-            Sampler sampl = new BatchSampler(new SequenceSampler(), 1, false);
-            int size = Math.toIntExact(size());
-            Number[][] data = new Number[size][];
-            Number[][] labels = new Number[size][];
-            int index = 0;
-            for (Batch batch : this.getData(manager, sampl)) {
-                data[index] = flattenRecord(batch.getData());
-                labels[index] = flattenRecord(batch.getLabels());
-                batch.close();
-                index++;
-            }
-            return new Pair<>(data, labels);
+    public Pair<Number[][], Number[][]> toArray(NDManager manager)
+            throws IOException, TranslateException {
+        Sampler sampl = new BatchSampler(new SequenceSampler(), 1, false);
+        int size = Math.toIntExact(size());
+        Number[][] data = new Number[size][];
+        Number[][] labels = new Number[size][];
+        int index = 0;
+        for (Batch batch : getData(manager, sampl)) {
+            data[index] = flattenRecord(batch.getData());
+            labels[index] = flattenRecord(batch.getLabels());
+            batch.close();
+            index++;
         }
+        return new Pair<>(data, labels);
     }
 
     private Number[] flattenRecord(NDList data) {
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index d39e07232..1500c6db4 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -70,7 +70,6 @@ nav:
           - BERT with MXNet: 'jupyter/mxnet/load_your_own_mxnet_bert.ipynb'
           - BERT with PyTorch: 'jupyter/pytorch/load_your_own_pytorch_bert.ipynb'
   - Guides:
-      - 'docs/engine.md'
       - Models:
           - 'docs/load_model.md'
           - 'docs/model-zoo.md'
@@ -137,12 +136,15 @@ nav:
   - Extensions:
       - Android: 'android/README.md'
       - AWS S3 support: 'extensions/aws-ai/README.md'
+      - Audio: 'extensions/audio/README.md'
       - fastText: 'extensions/fasttext/README.md'
       - Hadoop support: 'extensions/hadoop/README.md'
       - Huggingface Tokenizers: 'extensions/tokenizers/README.md'
       - OpenCV: 'extensions/opencv/README.md'
       - SentencePiece: 'extensions/sentencepiece/README.md'
+      - Spark support: 'extensions/spark/README.md'
       - Tablesaw: 'extensions/tablesaw/README.md'
+      - TimeSeries: 'extensions/timeseries/README.md'
       - DJL Zero: 'djl-zero/README.md'
       - DJL Benchmark: 'docs/serving/benchmark/README.md'
   - DJL Serving:
diff --git a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDArray.java b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDArray.java
index 8b38c7cb5..2ea08924d 100644
--- a/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDArray.java
+++ b/engines/ml/lightgbm/src/main/java/ai/djl/ml/lightgbm/LgbmNDArray.java
@@ -164,6 +164,7 @@ public class LgbmNDArray extends NDArrayAdapter {
         floatData = array.floatData;
         doubleData = array.doubleData;
         typeConstant = array.typeConstant;
+        replaced.close();
     }
 
     /** {@inheritDoc} */
diff --git a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArray.java b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArray.java
index 148be97e9..b3563f063 100644
--- a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArray.java
+++ b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArray.java
@@ -399,8 +399,8 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
         params.add("value", value);
         params.add("use_sequence_length", true);
         params.add("axis", 1);
-        return manager.invoke("_npx_sequence_mask", new NDList(this, sequenceLength), params)
-                .head();
+        NDList mask = new NDList(this, manager.from(sequenceLength));
+        return manager.invoke("_npx_sequence_mask", mask, params).head();
     }
 
     /** {@inheritDoc} */
@@ -430,6 +430,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
         if (getDataType() != other.getDataType()) {
             return false;
         }
+        other = manager.from(other);
         try (NDArray result = eq(other).toType(DataType.INT32, false)) {
             return result.all().getBoolean();
         }
@@ -446,6 +447,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray eq(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_equal", new NDArray[] {this, other}, null);
     }
 
@@ -460,6 +462,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray neq(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_not_equal", new NDArray[] {this, other}, null);
     }
 
@@ -474,6 +477,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray gt(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_greater", new NDArray[] {this, other}, null);
     }
 
@@ -488,6 +492,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray gte(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_greater_equal", new NDArray[] {this, other}, null);
     }
 
@@ -502,6 +507,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray lt(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_less", new NDArray[] {this, other}, null);
     }
 
@@ -516,6 +522,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray lte(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_less_equal", new NDArray[] {this, other}, null);
     }
 
@@ -530,6 +537,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray add(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_add", new NDArray[] {this, other}, null);
     }
 
@@ -544,6 +552,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray sub(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_subtract", new NDArray[] {this, other}, null);
     }
 
@@ -558,6 +567,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray mul(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_multiply", new NDArray[] {this, other}, null);
     }
 
@@ -592,6 +602,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray div(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_true_divide", new NDArray[] {this, other}, null);
     }
 
@@ -606,6 +617,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray mod(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_mod", new NDArray[] {this, other}, null);
     }
 
@@ -620,6 +632,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray pow(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_power", new NDArray[] {this, other}, null);
     }
 
@@ -635,6 +648,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray addi(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_add", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -651,6 +665,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray subi(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_subtract", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -667,6 +682,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray muli(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_multiply", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -684,6 +700,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray divi(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_true_divide", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -700,6 +717,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray modi(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_mod", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -716,6 +734,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray powi(NDArray other) {
+        other = manager.from(other);
         manager.invoke("_npi_power", new NDArray[] {this, other}, new NDArray[] {this}, null);
         return this;
     }
@@ -919,6 +938,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray maximum(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_maximum", new NDArray[] {this, other}, null);
     }
 
@@ -933,6 +953,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray minimum(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npi_minimum", new NDArray[] {this, other}, null);
     }
 
@@ -1148,6 +1169,9 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray squeeze(int[] axes) {
+        if (axes.length == 0) {
+            return squeeze();
+        }
         MxOpParams params = new MxOpParams();
         params.addTupleParam("axis", axes);
         return manager.invoke("_np_squeeze", this, params);
@@ -1163,6 +1187,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
                 (other.getDataType() == DataType.BOOLEAN)
                         ? other.toType(DataType.INT32, false)
                         : other;
+        other = manager.from(other);
         return manager.invoke("broadcast_logical_and", new NDArray[] {thisArr, other}, null)
                 .toType(DataType.BOOLEAN, false);
     }
@@ -1177,6 +1202,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
                 (other.getDataType() == DataType.BOOLEAN)
                         ? other.toType(DataType.INT32, false)
                         : other;
+        other = manager.from(other);
         return manager.invoke("broadcast_logical_or", new NDArray[] {thisArr, other}, null)
                 .toType(DataType.BOOLEAN, false);
     }
@@ -1191,6 +1217,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
                 (other.getDataType() == DataType.BOOLEAN)
                         ? other.toType(DataType.INT32, false)
                         : other;
+        other = manager.from(other);
         return manager.invoke("broadcast_logical_xor", new NDArray[] {thisArr, other}, null)
                 .toType(DataType.BOOLEAN, false);
     }
@@ -1396,6 +1423,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray dot(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_np_dot", new NDArray[] {this, other}, null);
     }
 
@@ -1405,6 +1433,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
         if (isScalar() || other.isScalar()) {
             throw new IllegalArgumentException("scalar is not allowed for matMul()");
         }
+        other = manager.from(other);
         return manager.invoke("_npi_matmul", new NDArray[] {this, other}, null);
     }
 
@@ -1572,6 +1601,7 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public NDArray batchDot(NDArray other) {
+        other = manager.from(other);
         return manager.invoke("_npx_batch_dot", new NDArray[] {this, other}, null);
     }
 
@@ -1639,8 +1669,8 @@ public class MxNDArray extends NativeResource<Pointer> implements LazyNDArray {
     /** {@inheritDoc} */
     @Override
     public boolean equals(Object obj) {
-        if (obj instanceof MxNDArray) {
-            return contentEquals((MxNDArray) obj);
+        if (obj instanceof NDArray) {
+            return contentEquals((NDArray) obj);
         }
         return false;
     }
diff --git a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayEx.java b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayEx.java
index b3d85dff5..48af22dc0 100644
--- a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayEx.java
+++ b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayEx.java
@@ -105,6 +105,7 @@ class MxNDArrayEx implements NDArrayEx {
     /** {@inheritDoc} */
     @Override
     public NDArray rdivi(NDArray b) {
+        b = getManager().from(b);
         getManager().invoke("elemwise_div", new NDArray[] {b, array}, new NDArray[] {array}, null);
         return array;
     }
@@ -160,6 +161,7 @@ class MxNDArrayEx implements NDArrayEx {
     /** {@inheritDoc} */
     @Override
     public NDArray rmodi(NDArray b) {
+        b = getManager().from(b);
         getManager().invoke("_npi_mod", new NDArray[] {b, array}, new NDArray[] {array}, null);
         return array;
     }
@@ -1060,7 +1062,11 @@ class MxNDArrayEx implements NDArrayEx {
         params.addParam("axis", axis);
         NDArray[] srcArray = new NDArray[arrays.size() + 1];
         srcArray[0] = array;
-        System.arraycopy(arrays.toArray(new NDArray[0]), 0, srcArray, 1, arrays.size());
+        NDManager manager = array.getManager();
+        int i = 1;
+        for (NDArray arr : arrays) {
+            srcArray[i++] = manager.from(arr);
+        }
         return getManager().invoke("_npi_stack", srcArray, params);
     }
 
@@ -1074,7 +1080,11 @@ class MxNDArrayEx implements NDArrayEx {
         params.addParam("axis", axis);
         NDArray[] srcArray = new NDArray[list.size() + 1];
         srcArray[0] = array;
-        System.arraycopy(list.toArray(new NDArray[0]), 0, srcArray, 1, list.size());
+        NDManager manager = array.getManager();
+        int i = 1;
+        for (NDArray arr : list) {
+            srcArray[i++] = manager.from(arr);
+        }
         return getManager().invoke("_npi_concatenate", srcArray, params);
     }
 
diff --git a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayIndexer.java b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayIndexer.java
index 6812aa4a1..b4aa2f084 100644
--- a/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayIndexer.java
+++ b/engines/mxnet/mxnet-engine/src/main/java/ai/djl/mxnet/engine/MxNDArrayIndexer.java
@@ -39,8 +39,8 @@ public class MxNDArrayIndexer extends NDArrayIndexer {
         params.addParam("axis", fullPick.getAxis());
         params.addParam("keepdims", true);
         params.add("mode", "wrap");
-        return manager.invoke("pick", new NDList(array, fullPick.getIndices()), params)
-                .singletonOrThrow();
+        NDList pick = new NDList(array, manager.from(fullPick.getIndices()));
+        return manager.invoke("pick", pick, params).singletonOrThrow();
     }
 
     /** {@inheritDoc} */
@@ -50,8 +50,8 @@ public class MxNDArrayIndexer extends NDArrayIndexer {
         MxOpParams params = new MxOpParams();
         params.addParam("axis", fullTake.getAxis());
         params.add("mode", "wrap");
-        return manager.invoke("take", new NDList(array, fullTake.getIndices()), params)
-                .singletonOrThrow();
+        NDList pick = new NDList(array, manager.from(fullTake.getIndices()));
+        return manager.invoke("take", pick, params).singletonOrThrow();
     }
 
     /** {@inheritDoc} */
diff --git a/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtNDArray.java b/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtNDArray.java
index 009266d56..0ba330980 100644
--- a/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtNDArray.java
+++ b/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtNDArray.java
@@ -58,6 +58,9 @@ public class OrtNDArray extends NDArrayAdapter {
     /** {@inheritDoc} */
     @Override
     public DataType getDataType() {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         if (dataType == null) {
             dataType = OrtUtils.toDataType(tensor.get().getInfo().type);
         }
@@ -67,6 +70,9 @@ public class OrtNDArray extends NDArrayAdapter {
     /** {@inheritDoc} */
     @Override
     public Shape getShape() {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         if (shape == null) {
             shape = new Shape(tensor.get().getInfo().getShape());
         }
@@ -94,6 +100,9 @@ public class OrtNDArray extends NDArrayAdapter {
     /** {@inheritDoc} */
     @Override
     public String[] toStringArray(Charset charset) {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         try {
             Object obj = tensor.get().getValue();
             if (obj instanceof String) {
diff --git a/engines/paddlepaddle/paddlepaddle-engine/src/main/java/ai/djl/paddlepaddle/engine/PpNDArray.java b/engines/paddlepaddle/paddlepaddle-engine/src/main/java/ai/djl/paddlepaddle/engine/PpNDArray.java
index f3fdc7f3f..0ffc901bf 100644
--- a/engines/paddlepaddle/paddlepaddle-engine/src/main/java/ai/djl/paddlepaddle/engine/PpNDArray.java
+++ b/engines/paddlepaddle/paddlepaddle-engine/src/main/java/ai/djl/paddlepaddle/engine/PpNDArray.java
@@ -82,6 +82,9 @@ public class PpNDArray extends NDArrayAdapter {
     /** {@inheritDoc} */
     @Override
     public DataType getDataType() {
+        if (isClosed) {
+            throw new IllegalStateException("Native resource has been release already.");
+        }
         if (dataType == null) {
             dataType = JniUtils.getDTypeFromNd(this);
         }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
index db91f276e..29148ec44 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArray.java
@@ -1103,11 +1103,11 @@ public class PtNDArray extends NativeResource<Long> implements NDArray {
     @Override
     public PtNDArray squeeze(int[] axes) {
         if (isScalar()) {
-            if (axes.length > 1 || axes[0] != 0) {
-                throw new IllegalArgumentException(
-                        "axis " + axes[0] + "is out of bounds for array of dimension 0");
+            if (axes.length == 0 || (axes.length == 1 && axes[0] == 0)) {
+                return (PtNDArray) duplicate();
             }
-            return (PtNDArray) duplicate();
+            throw new IllegalArgumentException(
+                    "axis " + axes[0] + " is out of bounds for array of dimension 0");
         }
         long[] shapeArr = getShape().getShape();
         List<Long> newShape = new ArrayList<>();
@@ -1567,8 +1567,8 @@ public class PtNDArray extends NativeResource<Long> implements NDArray {
     /** {@inheritDoc} */
     @Override
     public boolean equals(Object obj) {
-        if (obj instanceof PtNDArray) {
-            return contentEquals((PtNDArray) obj);
+        if (obj instanceof NDArray) {
+            return contentEquals((NDArray) obj);
         }
         return false;
     }
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayIndexer.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayIndexer.java
index b38a038d1..2f474bed2 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayIndexer.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayIndexer.java
@@ -91,7 +91,7 @@ public class PtNDArrayIndexer extends NDArrayIndexer {
         if (data instanceof Number) {
             JniUtils.indexAdvPut(ptArray, index, (PtNDArray) manager.create((Number) data));
         } else if (data instanceof NDArray) {
-            JniUtils.indexAdvPut(ptArray, index, (PtNDArray) data);
+            JniUtils.indexAdvPut(ptArray, index, manager.from((NDArray) data));
         } else {
             throw new IllegalArgumentException(
                     "The type of value to assign cannot be other than NDArray and Number.");
diff --git a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
index 3edbd3d99..f0b2d1831 100644
--- a/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
+++ b/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/JniUtils.java
@@ -396,7 +396,7 @@ public final class JniUtils {
                 PtNDArray indexArr = (PtNDArray) ((NDIndexBooleans) elem).getIndex();
                 PyTorchLibrary.LIB.torchIndexAppendArray(torchIndexHandle, indexArr.getHandle());
             } else if (elem instanceof NDIndexTake) {
-                PtNDArray indexArr = (PtNDArray) ((NDIndexTake) elem).getIndex();
+                PtNDArray indexArr = manager.from(((NDIndexTake) elem).getIndex());
                 if (indexArr.getDataType() != DataType.INT64) {
                     indexArr = indexArr.toType(DataType.INT64, true);
                 }
diff --git a/integration/build.gradle b/integration/build.gradle
index 0cb57bb6a..675d59061 100644
--- a/integration/build.gradle
+++ b/integration/build.gradle
@@ -16,11 +16,9 @@ dependencies {
     runtimeOnly project(":engines:tensorflow:tensorflow-model-zoo")
     runtimeOnly project(":engines:ml:xgboost")
     runtimeOnly project(":engines:ml:lightgbm")
+    runtimeOnly project(":engines:onnxruntime:onnxruntime-engine")
 
-    if (System.getProperty("ai.djl.default_engine") == "OnnxRuntime") {
-        // onnxruntime requires user install libgomp.so.1 manually, exclude from default dependency
-        runtimeOnly project(":engines:onnxruntime:onnxruntime-engine")
-    } else if (System.getProperty("ai.djl.default_engine") == "TFLite") {
+    if (System.getProperty("ai.djl.default_engine") == "TFLite") {
         runtimeOnly project(":engines:tflite:tflite-engine")
     } else if (System.getProperty("ai.djl.default_engine") == "DLR") {
         runtimeOnly project(":engines:dlr:dlr-engine")


Test patch:
diff --git a/integration/src/main/java/ai/djl/integration/tests/inference/PredictorTest.java b/integration/src/main/java/ai/djl/integration/tests/inference/PredictorTest.java
index 960a3aa9c..896407e49 100644
--- a/integration/src/main/java/ai/djl/integration/tests/inference/PredictorTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/inference/PredictorTest.java
@@ -16,6 +16,7 @@ import ai.djl.Device;
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.basic.Mlp;
 import ai.djl.inference.Predictor;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.Shape;
@@ -49,18 +50,18 @@ public class PredictorTest {
         predictWithDeviceHelper(Device.gpu(), Device.cpu());
     }
 
-    public void predictWithDeviceHelper(Device modelDevice, Device predictorDevice)
+    public void predictWithDeviceHelper(Device device, Device predictorDevice)
             throws TranslateException {
         // Create simple model on modelDevice
-        try (NDManager manager = NDManager.newBaseManager(modelDevice)) {
+        try (NDManager manager = NDManager.newBaseManager(device, TestUtils.getEngine());
+                Model model = Model.newInstance("mlp", device, TestUtils.getEngine())) {
             Block block = new Mlp(10, 10, new int[] {10});
-            Model model = Model.newInstance("mlp");
             model.setBlock(block);
 
             // Use trainer for initialization
             TrainingConfig config =
                     new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                            .optDevices(new Device[] {modelDevice});
+                            .optDevices(new Device[] {device});
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(1, 10));
             }
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/BufferedImageFactoryTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/BufferedImageFactoryTest.java
index f7420c750..fcc9fd3a1 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/BufferedImageFactoryTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/BufferedImageFactoryTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.modality.cv;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.ImageFactory;
 import ai.djl.ndarray.NDArray;
@@ -26,9 +27,10 @@ import org.testng.annotations.Test;
 import java.io.IOException;
 
 public class BufferedImageFactoryTest {
+
     @Test
     public void testLoadImage() throws IOException {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             ImageFactory factory = ImageFactory.getInstance();
             Image img =
                     factory.fromUrl(
@@ -40,7 +42,7 @@ public class BufferedImageFactoryTest {
 
     @Test
     public void testFormNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(0.0f, 12.0f).reshape(3, 2, 2);
             ImageFactory factory = ImageFactory.getInstance();
             Image image = factory.fromNDArray(array);
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxDetectionTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxDetectionTest.java
index 593890688..3fd1fa1c4 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxDetectionTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxDetectionTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.modality.cv;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.MultiBoxDetection;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
@@ -22,10 +23,11 @@ import ai.djl.testing.Assertions;
 import org.testng.annotations.Test;
 
 public class MultiBoxDetectionTest {
+
     @Test
     public void testDetections() {
         MultiBoxDetection multiBoxDetection = MultiBoxDetection.builder().build();
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray anchors =
                     manager.create(
                                     new float[] {
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxPriorTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxPriorTest.java
index f014bd681..cdbab1ca7 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxPriorTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxPriorTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.modality.cv;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.MultiBoxPrior;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
@@ -24,9 +25,10 @@ import java.util.Arrays;
 import java.util.List;
 
 public class MultiBoxPriorTest {
+
     @Test
     public void testMultiBoxPriors() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             List<Float> sizes = Arrays.asList(0.2f, 0.272f);
             List<Float> ratios = Arrays.asList(1f, 2f, 0.5f);
             MultiBoxPrior multiBoxPriors =
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxTargetTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxTargetTest.java
index c01666b85..e6e98469b 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxTargetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/MultiBoxTargetTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.modality.cv;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.MultiBoxTarget;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
@@ -23,9 +24,10 @@ import org.testng.Assert;
 import org.testng.annotations.Test;
 
 public class MultiBoxTargetTest {
+
     @Test
     public void testTargets() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray anchorBoxes = manager.arange(22840.0f * 4.0f).reshape(new Shape(1, 22840, 4));
             NDArray label = manager.arange(160.0f).reshape(new Shape(32, 1, 5));
             NDArray classPreds =
@@ -42,7 +44,8 @@ public class MultiBoxTargetTest {
 
     @Test
     public void testMultiBoxTargetValues() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray groundTruth =
                     manager.create(
                                     new float[] {
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/NDImageUtilsTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/NDImageUtilsTest.java
index 844d56dfe..e5822942c 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/NDImageUtilsTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/NDImageUtilsTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.modality.cv;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.util.NDImageUtils;
 import ai.djl.ndarray.NDArray;
@@ -20,7 +21,6 @@ import ai.djl.ndarray.index.NDIndex;
 import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.testing.Assertions;
-import ai.djl.testing.TestRequirements;
 import ai.djl.util.cuda.CudaUtils;
 
 import org.testng.Assert;
@@ -30,9 +30,10 @@ public class NDImageUtilsTest {
 
     @Test
     public void testNormalize() {
-        TestRequirements.notEngine("TensorFlow"); // TensorFlow use channels last by default
+        // TensorFlow use channels last by default
+        TestUtils.requiresEngine("MXNet", "PyTorch");
 
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test 3D C, H, W
             NDArray image = manager.ones(new Shape(3, 4, 2));
             float[] mean = {0.3f, 0.4f, 0.5f};
@@ -67,9 +68,10 @@ public class NDImageUtilsTest {
 
     @Test
     public void testToTensor() {
-        TestRequirements.notEngine("TensorFlow"); // TensorFlow use channels last by default
+        // TensorFlow use channels last by default
+        TestUtils.requiresEngine("MXNet", "PyTorch");
 
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test 3D C, H, W
             NDArray image = manager.randomUniform(0, 255, new Shape(4, 2, 3));
             NDArray toTensor = NDImageUtils.toTensor(image);
@@ -96,7 +98,9 @@ public class NDImageUtilsTest {
 
     @Test
     public void testResize() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Image.Interpolation[] interpolations = {
                 Image.Interpolation.NEAREST,
                 Image.Interpolation.BILINEAR,
@@ -147,7 +151,7 @@ public class NDImageUtilsTest {
 
     @Test
     public void testCrop() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test 3D H, W, C
             NDArray image = manager.randomUniform(0, 255, new Shape(100, 50, 3));
             NDArray result = NDImageUtils.crop(image, 10, 20, 30, 40);
@@ -170,7 +174,7 @@ public class NDImageUtilsTest {
 
     @Test
     public void testRandomCrop() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // make sure runnable
             NDArray image = manager.randomUniform(0, 255, new Shape(100, 50, 3));
             NDImageUtils.randomResizedCrop(image, 10, 10, 0.3, 0.7, 0.5, 2);
@@ -179,7 +183,7 @@ public class NDImageUtilsTest {
 
     @Test
     public void testRandomFlip() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             manager.getEngine().setRandomSeed(1234);
             // flip lr
             NDArray image = manager.randomUniform(0, 255, new Shape(10, 10, 3));
@@ -193,7 +197,7 @@ public class NDImageUtilsTest {
 
     @Test
     public void testRandomColor() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // make sure runnable
             NDArray image = manager.randomUniform(0, 255, new Shape(10, 10, 3));
             NDImageUtils.randomBrightness(image, 0.6f);
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/cv/SingleShotDetectionTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/cv/SingleShotDetectionTest.java
index ecee193ab..95428e61b 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/cv/SingleShotDetectionTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/cv/SingleShotDetectionTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.modality.cv;
 
 import ai.djl.basicmodelzoo.cv.object_detection.ssd.SingleShotDetection;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.DataType;
@@ -64,7 +65,7 @@ public class SingleShotDetectionTest {
 
     @Test
     public void testSingleShotDetectionShape() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int[] numFilters = {16, 32, 64};
             SequentialBlock block = new SequentialBlock();
             for (int numFilter : numFilters) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/modality/nlp/SimpleTextEncoderTest.java b/integration/src/main/java/ai/djl/integration/tests/modality/nlp/SimpleTextEncoderTest.java
index e725f3aa2..95e454b19 100644
--- a/integration/src/main/java/ai/djl/integration/tests/modality/nlp/SimpleTextEncoderTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/modality/nlp/SimpleTextEncoderTest.java
@@ -50,7 +50,7 @@ public class SimpleTextEncoderTest {
                                 .optBatchFirst(true)
                                 .optReturnState(true)
                                 .build());
-        try (NDManager manager = NDManager.newBaseManager(TestUtils.getDevices()[0])) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getDevices(1)[0])) {
             encoder.initialize(manager, DataType.FLOAT32, new Shape(4, 7));
             NDList output =
                     encoder.forward(
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/CustomTranslatorTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/CustomTranslatorTest.java
index bc86d2c5a..d651311f0 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/CustomTranslatorTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/CustomTranslatorTest.java
@@ -15,8 +15,8 @@ package ai.djl.integration.tests.model_zoo;
 import ai.djl.Application;
 import ai.djl.MalformedModelException;
 import ai.djl.ModelException;
-import ai.djl.engine.Engine;
 import ai.djl.inference.Predictor;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.Classifications;
 import ai.djl.modality.Classifications.Classification;
 import ai.djl.modality.Input;
@@ -31,7 +31,6 @@ import ai.djl.ndarray.NDManager;
 import ai.djl.repository.zoo.Criteria;
 import ai.djl.repository.zoo.ModelNotFoundException;
 import ai.djl.repository.zoo.ZooModel;
-import ai.djl.testing.TestRequirements;
 import ai.djl.translate.TranslateException;
 import ai.djl.translate.Translator;
 import ai.djl.util.JsonUtils;
@@ -66,7 +65,7 @@ public class CustomTranslatorTest {
 
     @BeforeClass
     public void setup() throws IOException, ModelNotFoundException, MalformedModelException {
-        if (!"MXNet".equals(Engine.getDefaultEngineName())) {
+        if (!"MXNet".equals(TestUtils.getEngine())) {
             return; // do not throw exception here
         }
 
@@ -76,6 +75,7 @@ public class CustomTranslatorTest {
                 Criteria.builder()
                         .setTypes(Image.class, Classifications.class)
                         .optArtifactId("ai.djl.mxnet:mlp")
+                        .optEngine(TestUtils.getEngine())
                         .build();
 
         try (ZooModel<Image, Classifications> model = criteria.loadModel()) {
@@ -109,7 +109,7 @@ public class CustomTranslatorTest {
     @Test
     public void testImageClassificationTranslator()
             throws IOException, ModelException, TranslateException {
-        TestRequirements.engine("MXNet");
+        TestUtils.requiresEngine("MXNet");
 
         // load RawTranslator
         runRawTranslator();
@@ -159,12 +159,13 @@ public class CustomTranslatorTest {
 
     @Test
     public void testSsdTranslator() throws IOException, ModelException, TranslateException {
-        TestRequirements.engine("MXNet");
+        TestUtils.requiresEngine("MXNet");
 
         Criteria<Image, DetectedObjects> c =
                 Criteria.builder()
                         .setTypes(Image.class, DetectedObjects.class)
                         .optArtifactId("ai.djl.mxnet:ssd")
+                        .optEngine(TestUtils.getEngine())
                         .build();
         String modelUrl;
         try (ZooModel<Image, DetectedObjects> model = c.loadModel()) {
@@ -184,6 +185,7 @@ public class CustomTranslatorTest {
                                 "translatorFactory",
                                 "ai.djl.modality.cv.translator.SingleShotDetectionTranslatorFactory")
                         .optModelName("ssd_512_resnet50_v1_voc")
+                        .optEngine(TestUtils.getEngine())
                         .build();
 
         Path imageFile = Paths.get("../examples/src/test/resources/dog_bike_car.jpg");
@@ -214,6 +216,7 @@ public class CustomTranslatorTest {
                         .optApplication(application)
                         .optArguments(arguments)
                         .optModelPath(modelDir)
+                        .optEngine(TestUtils.getEngine())
                         .build();
 
         try (ZooModel<Input, Output> model = criteria.loadModel();
@@ -232,11 +235,12 @@ public class CustomTranslatorTest {
         }
     }
 
-    public void runRawTranslator() throws IOException, ModelException, TranslateException {
+    private void runRawTranslator() throws IOException, ModelException, TranslateException {
         Criteria<Input, Output> criteria =
                 Criteria.builder()
                         .setTypes(Input.class, Output.class)
                         .optModelPath(modelDir)
+                        .optEngine(TestUtils.getEngine())
                         .build();
 
         try (ZooModel<Input, Output> model = criteria.loadModel();
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/ImperativeModelZooTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/ImperativeModelZooTest.java
index c86f73678..f3247b935 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/ImperativeModelZooTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/ImperativeModelZooTest.java
@@ -17,6 +17,7 @@ import ai.djl.MalformedModelException;
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.BasicModelZoo;
 import ai.djl.basicmodelzoo.cv.classification.AlexNet;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.Classifications;
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.output.DetectedObjects;
@@ -24,7 +25,6 @@ import ai.djl.ndarray.types.Shape;
 import ai.djl.nn.Block;
 import ai.djl.repository.zoo.Criteria;
 import ai.djl.repository.zoo.ModelNotFoundException;
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.DefaultTrainingConfig;
 import ai.djl.training.Trainer;
 import ai.djl.training.loss.SoftmaxCrossEntropyLoss;
@@ -39,7 +39,7 @@ public class ImperativeModelZooTest {
     @Test
     public void testImperativeModelInputOutput()
             throws MalformedModelException, ModelNotFoundException, IOException {
-        TestRequirements.engine("MXNet"); // Resnet50-cifar10 model only available in MXNet
+        TestUtils.requiresEngine("MXNet"); // Resnet50-cifar10 model only available in MXNet
 
         // from model zoo
         Criteria<Image, Classifications> criteria =
@@ -48,6 +48,7 @@ public class ImperativeModelZooTest {
                         .setTypes(Image.class, Classifications.class)
                         .optGroupId(BasicModelZoo.GROUP_ID)
                         .optArtifactId("resnet")
+                        .optEngine(TestUtils.getEngine())
                         .optFilter("layers", "50")
                         .optFilter("dataset", "cifar10")
                         .build();
@@ -60,6 +61,7 @@ public class ImperativeModelZooTest {
                 Criteria.builder()
                         .optApplication(Application.CV.OBJECT_DETECTION)
                         .setTypes(Image.class, DetectedObjects.class)
+                        .optEngine(TestUtils.getEngine())
                         .optGroupId(BasicModelZoo.GROUP_ID)
                         .build();
         try (Model model = ssdCriteria.loadModel()) {
@@ -71,7 +73,7 @@ public class ImperativeModelZooTest {
 
         // from builder
         Block alexNet = AlexNet.builder().build();
-        try (Model model = Model.newInstance("alexnet")) {
+        try (Model model = Model.newInstance("alexnet", TestUtils.getEngine())) {
             model.setBlock(alexNet);
             try (Trainer trainer =
                     model.newTrainer(new DefaultTrainingConfig(new SoftmaxCrossEntropyLoss()))) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/AlexNetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/AlexNetTest.java
index 537dc9442..5a97d58f4 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/AlexNetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/AlexNetTest.java
@@ -15,7 +15,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.AlexNet;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -46,10 +46,10 @@ public class AlexNetTest {
     public void testTrainWithDefaultChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
 
         Block alexNet = AlexNet.builder().build();
-        try (Model model = Model.newInstance("alexnet")) {
+        try (Model model = Model.newInstance("alexnet", TestUtils.getEngine())) {
             model.setBlock(alexNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -100,13 +100,13 @@ public class AlexNetTest {
     public void testTrainWithCustomChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block alexNet =
                 AlexNet.builder()
                         .setDropOutRate(0.8f)
                         .setNumChannels(new int[] {128, 128, 128, 512, 384, 2048, 2048})
                         .build();
-        try (Model model = Model.newInstance("alexnet")) {
+        try (Model model = Model.newInstance("alexnet", TestUtils.getEngine())) {
             model.setBlock(alexNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -155,7 +155,7 @@ public class AlexNetTest {
 
     @Test
     public void testOutputShapes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int batchSize = 2;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
             Shape currentShape = x.getShape();
@@ -186,7 +186,7 @@ public class AlexNetTest {
 
     @Test
     public void testForwardMethod() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block alexNet = AlexNet.builder().build();
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/GoogLeNetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/GoogLeNetTest.java
index 517920a14..3ed4dc5ff 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/GoogLeNetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/GoogLeNetTest.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.GoogLeNet;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -45,9 +45,9 @@ public class GoogLeNetTest {
     public void testTrainWithDefaultChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block googLeNet = GoogLeNet.builder().build();
-        try (Model model = Model.newInstance("googlenet")) {
+        try (Model model = Model.newInstance("googlenet", TestUtils.getEngine())) {
             model.setBlock(googLeNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -96,7 +96,7 @@ public class GoogLeNetTest {
 
     @Test
     public void testOutputShapes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 96, 96));
             Shape currentShape = x.getShape();
@@ -129,7 +129,7 @@ public class GoogLeNetTest {
 
     @Test
     public void testForwardMethod() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block googLeNet = GoogLeNet.builder().build();
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 28, 28));
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/LeNetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/LeNetTest.java
index 2bf10ce26..4999e35e6 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/LeNetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/LeNetTest.java
@@ -15,7 +15,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.LeNet;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -46,9 +46,9 @@ public class LeNetTest {
     public void testTrainWithDefaultChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block leNet = LeNet.builder().build();
-        try (Model model = Model.newInstance("lenet")) {
+        try (Model model = Model.newInstance("lenet", TestUtils.getEngine())) {
             model.setBlock(leNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -90,9 +90,9 @@ public class LeNetTest {
     public void testTrainWithCustomChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block leNet = LeNet.builder().setNumChannels(new int[] {12, 15, 150, 100}).build();
-        try (Model model = Model.newInstance("lenet")) {
+        try (Model model = Model.newInstance("lenet", TestUtils.getEngine())) {
             model.setBlock(leNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -133,7 +133,7 @@ public class LeNetTest {
 
     @Test
     public void testOutputShapes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 28, 28));
             Shape currentShape = x.getShape();
@@ -162,7 +162,7 @@ public class LeNetTest {
 
     @Test
     public void testForwardMethod() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block leNet = LeNet.builder().build();
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 28, 28));
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV1Test.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV1Test.java
index 0ab2f2a7c..456389ce8 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV1Test.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV1Test.java
@@ -15,7 +15,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.MobileNetV1;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -36,14 +36,15 @@ import ai.djl.util.PairList;
 import org.testng.annotations.Test;
 
 public class MobileNetV1Test {
+
     @Test
     public void testTrain() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
         Block mobilenet = MobileNetV1.builder().build();
-        try (Model model = Model.newInstance("mobilenet")) {
+        try (Model model = Model.newInstance("mobilenet", TestUtils.getEngine())) {
             model.setBlock(mobilenet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV2Test.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV2Test.java
index 8fc9baa59..985dd1034 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV2Test.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/MobileNetV2Test.java
@@ -15,7 +15,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.MobileNetV2;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -36,14 +36,15 @@ import ai.djl.util.PairList;
 import org.testng.annotations.Test;
 
 public class MobileNetV2Test {
+
     @Test
     public void testTrain() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
         Block mobilenet = MobileNetV2.builder().setOutSize(10).build();
-        try (Model model = Model.newInstance("mobilenet")) {
+        try (Model model = Model.newInstance("mobilenet", TestUtils.getEngine())) {
             model.setBlock(mobilenet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/NiNTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/NiNTest.java
index 4c4d9d0d9..8e4938971 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/NiNTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/NiNTest.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.NiN;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -45,9 +45,9 @@ public class NiNTest {
     public void testTrainWithDefaultChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block nin = NiN.builder().build();
-        try (Model model = Model.newInstance("nin")) {
+        try (Model model = Model.newInstance("nin", TestUtils.getEngine())) {
             model.setBlock(nin);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -97,13 +97,13 @@ public class NiNTest {
     public void testTrainWithCustomChannels() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
         Block nin =
                 NiN.builder()
                         .setDropOutRate(0.8f)
                         .setNumChannels(new int[] {48, 128, 384, 10})
                         .build();
-        try (Model model = Model.newInstance("nin")) {
+        try (Model model = Model.newInstance("nin", TestUtils.getEngine())) {
             model.setBlock(nin);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -148,7 +148,7 @@ public class NiNTest {
 
     @Test
     public void testOutputShapes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
             Shape currentShape = x.getShape();
@@ -180,7 +180,7 @@ public class NiNTest {
 
     @Test
     public void testForwardMethod() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block nin = NiN.builder().build();
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/ResnetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/ResnetTest.java
index c757c8dd3..e2b6711ae 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/ResnetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/ResnetTest.java
@@ -17,8 +17,8 @@ import ai.djl.MalformedModelException;
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.BasicModelZoo;
 import ai.djl.basicmodelzoo.cv.classification.ResNetV1;
-import ai.djl.engine.Engine;
 import ai.djl.inference.Predictor;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.Classifications;
 import ai.djl.modality.cv.Image;
 import ai.djl.ndarray.NDArray;
@@ -31,7 +31,6 @@ import ai.djl.repository.zoo.Criteria;
 import ai.djl.repository.zoo.ModelNotFoundException;
 import ai.djl.repository.zoo.ZooModel;
 import ai.djl.testing.Assertions;
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.DefaultTrainingConfig;
 import ai.djl.training.EasyTrain;
 import ai.djl.training.Trainer;
@@ -57,7 +56,7 @@ public class ResnetTest {
     public void testTrain() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block resNet50 =
@@ -66,7 +65,7 @@ public class ResnetTest {
                         .setNumLayers(50)
                         .setOutSize(10)
                         .build();
-        try (Model model = Model.newInstance("resnet")) {
+        try (Model model = Model.newInstance("resnet", TestUtils.getEngine())) {
             model.setBlock(resNet50);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -107,7 +106,7 @@ public class ResnetTest {
     public void testWithIntermediate() throws TranslateException {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block resNet50 =
@@ -118,7 +117,7 @@ public class ResnetTest {
                         .build()
                         .setReturnIntermediate(true);
 
-        try (Model model = Model.newInstance("resnet")) {
+        try (Model model = Model.newInstance("resnet", TestUtils.getEngine())) {
             model.setBlock(resNet50);
             int batchSize = 1;
             Shape inputShape = new Shape(batchSize, 1, 28, 28);
@@ -157,7 +156,7 @@ public class ResnetTest {
         try (ZooModel<Image, Classifications> model = getModel()) {
             TrainingConfig config =
                     new DefaultTrainingConfig(Loss.l1Loss())
-                            .optDevices(Engine.getInstance().getDevices(2))
+                            .optDevices(TestUtils.getDevices(2))
                             .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 2;
@@ -187,7 +186,7 @@ public class ResnetTest {
 
     private ZooModel<Image, Classifications> getModel()
             throws IOException, ModelNotFoundException, MalformedModelException {
-        TestRequirements.engine("MXNet"); // Resnet50-cifar10 model only available in MXNet
+        TestUtils.requiresEngine("MXNet"); // Resnet50-cifar10 model only available in MXNet
 
         Criteria<Image, Classifications> criteria =
                 Criteria.builder()
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/SqueezenetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/SqueezenetTest.java
index 2daa60696..c6008bff0 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/SqueezenetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/SqueezenetTest.java
@@ -15,7 +15,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.SqueezeNet;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -41,10 +41,10 @@ public class SqueezenetTest {
     public void testTrain() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
         Block squeezeNet = SqueezeNet.squeezenet(10);
-        try (Model model = Model.newInstance("squeezenet")) {
+        try (Model model = Model.newInstance("squeezenet", TestUtils.getEngine())) {
             model.setBlock(squeezeNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/VGGTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/VGGTest.java
index 76c7d3dbf..a9a43257b 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/VGGTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/classification/VGGTest.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.model_zoo.classification;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.classification.VGG;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -50,10 +50,10 @@ public class VGGTest {
 
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
 
         Block vgg = VGG.builder().build();
-        try (Model model = Model.newInstance("vgg")) {
+        try (Model model = Model.newInstance("vgg", TestUtils.getEngine())) {
             model.setBlock(vgg);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -102,7 +102,7 @@ public class VGGTest {
 
     @Test
     public void testOutputShapes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
             Shape currentShape = x.getShape();
@@ -137,7 +137,7 @@ public class VGGTest {
 
     @Test
     public void testForwardMethod() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block vgg = VGG.builder().build();
             int batchSize = 1;
             NDArray x = manager.ones(new Shape(batchSize, 1, 224, 224));
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/SingleShotDetectionTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/SingleShotDetectionTest.java
index 28cdfd2b3..b5907925e 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/SingleShotDetectionTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/SingleShotDetectionTest.java
@@ -14,11 +14,11 @@
 package ai.djl.integration.tests.model_zoo.object_detection;
 
 import ai.djl.Application;
-import ai.djl.MalformedModelException;
+import ai.djl.ModelException;
 import ai.djl.basicdataset.cv.PikachuDetection;
 import ai.djl.basicmodelzoo.BasicModelZoo;
-import ai.djl.engine.Engine;
 import ai.djl.inference.Predictor;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.cv.Image;
 import ai.djl.modality.cv.ImageFactory;
 import ai.djl.modality.cv.MultiBoxDetection;
@@ -31,9 +31,7 @@ import ai.djl.nn.Block;
 import ai.djl.nn.LambdaBlock;
 import ai.djl.nn.SequentialBlock;
 import ai.djl.repository.zoo.Criteria;
-import ai.djl.repository.zoo.ModelNotFoundException;
 import ai.djl.repository.zoo.ZooModel;
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.DefaultTrainingConfig;
 import ai.djl.training.EasyTrain;
 import ai.djl.training.Trainer;
@@ -56,9 +54,7 @@ import java.nio.file.Paths;
 public class SingleShotDetectionTest {
 
     @Test
-    public void testLoadPredict()
-            throws IOException, ModelNotFoundException, TranslateException,
-                    MalformedModelException {
+    public void testLoadPredict() throws IOException, ModelException, TranslateException {
         try (ZooModel<Image, DetectedObjects> model = getModel()) {
             model.setBlock(getPredictBlock(model.getBlock()));
             try (Predictor<Image, DetectedObjects> predictor = model.newPredictor()) {
@@ -73,9 +69,7 @@ public class SingleShotDetectionTest {
     }
 
     @Test
-    public void testLoadTrain()
-            throws IOException, ModelNotFoundException, MalformedModelException,
-                    TranslateException {
+    public void testLoadTrain() throws IOException, ModelException, TranslateException {
         try (ZooModel<Image, DetectedObjects> model = getModel()) {
             TrainingConfig config = setupTrainingConfig();
             try (Trainer trainer = model.newTrainer(config)) {
@@ -125,12 +119,13 @@ public class SingleShotDetectionTest {
         return new DefaultTrainingConfig(new SingleShotDetectionLoss())
                 .addEvaluator(new SingleShotDetectionAccuracy("classAccuracy"))
                 .addEvaluator(new BoundingBoxError("boundingBoxError"))
-                .optDevices(Engine.getInstance().getDevices(1));
+                .optDevices(TestUtils.getDevices(1));
     }
 
-    private ZooModel<Image, DetectedObjects> getModel()
-            throws IOException, ModelNotFoundException, MalformedModelException {
-        TestRequirements.engine("MXNet"); // SSD-pikachu model only available in MXNet
+    private ZooModel<Image, DetectedObjects> getModel() throws IOException, ModelException {
+        // SSD-pikachu model only available in MXNet
+        // TODO: Add PyTorch model to model zoo
+        TestUtils.requiresEngine("MXNet");
 
         Criteria<Image, DetectedObjects> criteria =
                 Criteria.builder()
@@ -138,6 +133,7 @@ public class SingleShotDetectionTest {
                         .setTypes(Image.class, DetectedObjects.class)
                         .optGroupId(BasicModelZoo.GROUP_ID)
                         .optArtifactId("ssd")
+                        .optEngine(TestUtils.getEngine())
                         .optFilter("flavor", "tiny")
                         .optFilter("dataset", "pikachu")
                         .build();
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/YOLOv3Test.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/YOLOv3Test.java
index 3eef4f6c6..6ac3cc918 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/YOLOv3Test.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/object_detection/YOLOv3Test.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.model_zoo.object_detection;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.cv.object_detection.yolo.YOLOV3;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -22,7 +22,6 @@ import ai.djl.ndarray.types.Shape;
 import ai.djl.nn.Block;
 import ai.djl.nn.Parameter;
 import ai.djl.testing.Assertions;
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.DefaultTrainingConfig;
 import ai.djl.training.EasyTrain;
 import ai.djl.training.Trainer;
@@ -38,14 +37,15 @@ import org.testng.Assert;
 import org.testng.annotations.Test;
 
 public class YOLOv3Test {
+
     @Test
     public void testDarkNet() {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
         Block darkNet = YOLOV3.builder().buildDarkNet();
-        try (Model model = Model.newInstance("darkNet")) {
+        try (Model model = Model.newInstance("darkNet", TestUtils.getEngine())) {
             model.setBlock(darkNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 1;
@@ -96,7 +96,7 @@ public class YOLOv3Test {
 
     @Test
     public void testYoloV3() {
-        TestRequirements.engine("PyTorch");
+        TestUtils.requiresEngine("PyTorch");
 
         float[] anchorsArray = YOLOv3Loss.getPresetAnchors();
         for (int i = 0; i < anchorsArray.length; i++) {
@@ -109,11 +109,11 @@ public class YOLOv3Test {
                                         .setInputShape(new Shape(256, 256))
                                         .setAnchorsArray(anchorsArray)
                                         .build())
-                        .optDevices(Engine.getInstance().getDevices(2))
+                        .optDevices(TestUtils.getDevices(2))
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block yolov3 = YOLOV3.builder().setNumClasses(1).build();
-        try (Model model = Model.newInstance("yolov3")) {
+        try (Model model = Model.newInstance("yolov3", TestUtils.getEngine())) {
             model.setBlock(yolov3);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 4;
diff --git a/integration/src/main/java/ai/djl/integration/tests/model_zoo/tabular/TabNetTest.java b/integration/src/main/java/ai/djl/integration/tests/model_zoo/tabular/TabNetTest.java
index de66efd2d..57c16822a 100644
--- a/integration/src/main/java/ai/djl/integration/tests/model_zoo/tabular/TabNetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/model_zoo/tabular/TabNetTest.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.model_zoo.tabular;
 
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.tabular.TabNet;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -42,7 +42,7 @@ public class TabNetTest {
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(TabNet.tabNetGLUBlock(1));
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -62,10 +62,10 @@ public class TabNetTest {
     public void testTrainingAndLogic() {
         TrainingConfig config =
                 new DefaultTrainingConfig(new TabNetRegressionLoss())
-                        .optDevices(Engine.getInstance().getDevices(2));
+                        .optDevices(TestUtils.getDevices(2));
 
         Block tabNet = TabNet.builder().setOutDim(10).build();
-        try (Model model = Model.newInstance("tabNet")) {
+        try (Model model = Model.newInstance("tabNet", TestUtils.getEngine())) {
             model.setBlock(tabNet);
             try (Trainer trainer = model.newTrainer(config)) {
                 int batchSize = 32;
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayAttachmentTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayAttachmentTest.java
index c5526d355..c3900a10e 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayAttachmentTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayAttachmentTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.index.NDIndex;
@@ -24,9 +25,9 @@ public class NDArrayAttachmentTest {
 
     @Test
     public void testReturnResource() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array3x4 = manager.ones(new Shape(3, 4));
-            try (NDManager subManager = NDManager.newBaseManager()) {
+            try (NDManager subManager = NDManager.newBaseManager(TestUtils.getEngine())) {
                 array3x4.tempAttach(subManager);
                 Assert.assertEquals(array3x4.getManager(), subManager);
             }
@@ -36,12 +37,15 @@ public class NDArrayAttachmentTest {
 
     @Test
     public void testIndexationUsesSpecificManager() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        // Hybrid engine's operation will change its' NDManager
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array3x4 = manager.ones(new Shape(3, 4));
             array3x4.setName("Test()");
             NDArray array4 = array3x4.get(1);
             Assert.assertEquals(array4.getManager(), manager);
-            try (NDManager subManager = NDManager.newBaseManager()) {
+            try (NDManager subManager = NDManager.newBaseManager(TestUtils.getEngine())) {
                 NDArray array4sub1 = array3x4.get(subManager, 1);
                 Assert.assertEquals(array4sub1.getManager(), subManager);
                 NDArray array4sub2 = array3x4.get(subManager, new NDIndex(1));
@@ -52,7 +56,7 @@ public class NDArrayAttachmentTest {
 
     @Test(expectedExceptions = IllegalStateException.class)
     public void testCannotUseCappedManager() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             manager.cap();
             manager.ones(new Shape(3, 4));
         }
@@ -60,7 +64,7 @@ public class NDArrayAttachmentTest {
 
     @Test(expectedExceptions = IllegalStateException.class)
     public void testIndexationCannotUseCappedManager() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array3x4 = manager.ones(new Shape(3, 4));
             array3x4.setName("Test()");
             manager.cap();
@@ -70,11 +74,14 @@ public class NDArrayAttachmentTest {
 
     @Test
     public void testIndexationUsesSpecificUncappedManager() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        // Hybrid engine's operation will change its' NDManager
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array3x4 = manager.ones(new Shape(3, 4));
             array3x4.setName("Test()");
             manager.cap();
-            try (NDManager subManager = NDManager.newBaseManager()) {
+            try (NDManager subManager = NDManager.newBaseManager(TestUtils.getEngine())) {
                 NDArray array4sub1 = array3x4.get(subManager, 1);
                 Assert.assertEquals(array4sub1.getManager(), subManager);
                 array3x4.tempAttach(subManager);
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayCreationOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayCreationOpTest.java
index 853142cb5..2d3ff7e7a 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayCreationOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayCreationOpTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.ndarray;
 
 import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDList;
@@ -21,7 +22,6 @@ import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.ndarray.types.SparseFormat;
 import ai.djl.testing.Assertions;
-import ai.djl.testing.TestRequirements;
 import ai.djl.util.Pair;
 import ai.djl.util.PairList;
 
@@ -35,7 +35,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testCreation() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // regular case
             NDArray array = manager.create(new float[] {0, 1, 2, 3}, new Shape(2, 2));
             Assert.assertEquals(array.toFloatArray(), new float[] {0, 1, 2, 3});
@@ -70,7 +70,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testCreateCSRMatrix() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             float[] expected = {7, 8, 9};
             FloatBuffer buf = FloatBuffer.wrap(expected);
             long[] indptr = {0, 2, 2, 3};
@@ -86,7 +86,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testCreateRowSparseMatrix() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             float[] expected = {1, 2, 3, 4, 5, 6};
             FloatBuffer buf = FloatBuffer.wrap(expected);
             long[] indices = {0, 1, 3};
@@ -104,7 +104,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testCreateCooMatrix() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             long[][] indices = {{0, 1, 1}, {2, 0, 2}};
             float[] values = {3, 4, 5};
             FloatBuffer buf = FloatBuffer.wrap(values);
@@ -118,7 +118,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testCreateNDArrayAndConvertToSparse() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray nd = manager.ones(new Shape(3, 5));
             try {
                 // Only MXNet support CSR
@@ -139,7 +139,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testDuplicate() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.zeros(new Shape(5));
             NDArray expected = manager.create(new float[] {0f, 0f, 0f, 0f, 0f});
             NDArray duplicate = array.duplicate();
@@ -172,7 +172,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testZeros() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.zeros(new Shape(5));
             NDArray expected = manager.create(new float[] {0f, 0f, 0f, 0f, 0f});
             Assertions.assertAlmostEquals(array, expected);
@@ -196,7 +196,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testOnes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.ones(new Shape(5));
             NDArray expected = manager.create(new float[] {1f, 1f, 1f, 1f, 1f});
             Assertions.assertAlmostEquals(array, expected);
@@ -220,7 +220,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testFull() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.full(new Shape(5), 3);
             NDArray expected = manager.create(new int[] {3, 3, 3, 3, 3});
             Assert.assertEquals(array, expected);
@@ -254,7 +254,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testZerosLike() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new Shape(5));
             NDArray expected = manager.create(new float[] {0f, 0f, 0f, 0f, 0f});
             Assert.assertEquals(array.zerosLike(), expected);
@@ -277,7 +277,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testOnesLike() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new Shape(5));
             NDArray expected = manager.create(new float[] {1f, 1f, 1f, 1f, 1f});
             Assert.assertEquals(array.onesLike(), expected);
@@ -300,7 +300,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testArange() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(0, 10, 1);
             NDArray expected = manager.create(new int[] {0, 1, 2, 3, 4, 5, 6, 7, 8, 9});
             Assert.assertEquals(array, expected);
@@ -334,7 +334,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testEye() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray expected = manager.eye(2);
             NDArray array = manager.create(new float[] {1f, 0f, 0f, 1f}, new Shape(2, 2));
             Assert.assertEquals(array, expected);
@@ -352,7 +352,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testLinspace() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.linspace(0.0f, 9.0f, 10, true, manager.getDevice());
             NDArray expected = manager.arange(10.0f);
             Assert.assertEquals(array, expected);
@@ -378,7 +378,7 @@ public class NDArrayCreationOpTest {
         testCases.add(0L, 2L);
         testCases.add(1000000L, 2000000L);
         testCases.add(-1234567L, 1234567L);
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             for (Pair<Long, Long> testCase : testCases) {
                 long low = testCase.getKey();
                 long high = testCase.getValue();
@@ -396,7 +396,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testRandomPermutation() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             long size = 3;
             NDArray array = manager.randomPermutation(size);
             Assert.assertTrue(
@@ -411,7 +411,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testRandomUniform() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray uniform = manager.randomUniform(0, 10, new Shape(1000, 1000));
             Assert.assertTrue(uniform.min().getFloat() >= 0f);
             Assert.assertTrue(uniform.max().getFloat() < 10f);
@@ -421,7 +421,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testRandomNormal() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray normal = manager.randomNormal(new Shape(1000, 1000));
             NDArray mean = normal.mean();
             NDArray std = normal.sub(mean).pow(2).mean();
@@ -432,7 +432,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testTruncatedNormal() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray normal = manager.truncatedNormal(new Shape(1000, 1000));
             Assertions.assertAlmostEquals(normal.mean().getFloat(), 0f, 2e-2f, 2e-2f);
             Assert.assertTrue(normal.gte(-2).all().getBoolean());
@@ -442,21 +442,21 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testFixedSeed() {
-        TestRequirements.notEngine(
-                "TensorFlow"); // TensorFlow fixed random seed requires restart process.
-
-        try (NDManager manager = NDManager.newBaseManager()) {
+        // TensorFlow fixed random seed requires restart process.
+        TestUtils.requiresEngine("MXNet", "PyTorch");
 
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             int fixedSeed = 1234;
-            Engine.getInstance().setRandomSeed(fixedSeed);
+            Engine engine = Engine.getEngine(TestUtils.getEngine());
+            engine.setRandomSeed(fixedSeed);
             NDArray expectedUniform = manager.randomUniform(-10, 10, new Shape(10, 10));
-            Engine.getInstance().setRandomSeed(fixedSeed);
+            engine.setRandomSeed(fixedSeed);
             NDArray actualUniform = manager.randomUniform(-10, 10, new Shape(10, 10));
             Assertions.assertAlmostEquals(expectedUniform, actualUniform, 1e-2f, 1e-2f);
 
-            Engine.getInstance().setRandomSeed(fixedSeed);
+            engine.setRandomSeed(fixedSeed);
             NDArray expectedNormal = manager.randomNormal(new Shape(100, 100));
-            Engine.getInstance().setRandomSeed(fixedSeed);
+            engine.setRandomSeed(fixedSeed);
             NDArray actualNormal = manager.randomNormal(new Shape(100, 100));
             Assertions.assertAlmostEquals(expectedNormal, actualNormal, 1e-2f, 1e-2f);
         }
@@ -464,7 +464,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testSamplePoisson() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray lam = manager.create(new double[] {1., 8.5});
 
             NDArray poisson = manager.samplePoisson(lam);
@@ -485,7 +485,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testSampleGamma() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray alpha = manager.create(new double[] {0., 2.5});
             NDArray beta = manager.create(new double[] {1., 0.7});
 
@@ -507,7 +507,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testSampleNormal() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray mu = manager.create(new double[] {0., 2.5});
             NDArray sigma = manager.create(new double[] {1., 3.7});
             NDArray normal = manager.sampleNormal(mu, sigma);
@@ -528,7 +528,7 @@ public class NDArrayCreationOpTest {
 
     @Test
     public void testHanningWindow() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray result = manager.hanningWindow(6);
             NDArray expected = manager.create(new float[] {0.0f, 0.25f, 0.75f, 1.0f, 0.75f, 0.25f});
             Assertions.assertAlmostEquals(result, expected);
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementArithmeticOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementArithmeticOpTest.java
index cb1d3b6ce..4fc2d0934 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementArithmeticOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementArithmeticOpTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.ndarray;
 
 import ai.djl.Model;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDManager;
@@ -133,7 +134,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testAddScalar() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
             NDManager manager = model.getNDManager();
             NDArray lhs = manager.create(new float[] {1f, 2f, 3f, 4f});
@@ -178,7 +179,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testAddNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray addend = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray addendum = manager.create(new float[] {2f, 3f, 4f, 5f});
             NDArray result = NDArrays.add(addend, addendum);
@@ -224,7 +225,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testSubScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray minuend = manager.create(new float[] {6, 9, 12, 11, 0});
             NDArray result = NDArrays.sub(minuend, 3);
             NDArray inPlaceResult = NDArrays.subi(minuend, 3);
@@ -240,7 +241,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testSubNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray minuend = manager.create(new float[] {6, 9, 12, 15, 0});
             NDArray subtrahend = manager.create(new float[] {2, 3, 4, 5, 6});
             NDArray result = NDArrays.sub(minuend, subtrahend);
@@ -259,7 +260,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseSubScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray minuend = manager.create(new float[] {6, 91, 12, 215, 180});
             NDArray result = NDArrays.sub(180, minuend);
             NDArray inPlaceResult = NDArrays.subi(180, minuend);
@@ -277,7 +278,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseSubNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray minuend = manager.create(new float[] {6, 9, 12, 15, 45});
             NDArray subtrahend = manager.create(new float[] {24, 63, 96, 15, 90});
             NDArray result = minuend.getNDArrayInternal().rsub(subtrahend);
@@ -319,7 +320,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testMulScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray multiplicand = manager.create(new float[] {6, 9, -12, 15, 0});
             NDArray result = NDArrays.mul(multiplicand, 3);
             NDArray expected = manager.create(new float[] {18, 27, -36, 45, 0});
@@ -340,7 +341,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testMulNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray multiplicand = manager.create(new float[] {6, 9, 12, 15, 0});
             NDArray with = manager.create(new float[] {2, 3, 4, 5, 6});
             NDArray result = NDArrays.mul(multiplicand, with);
@@ -373,7 +374,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testDot() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
             NDManager manager = model.getNDManager();
             NDArray lhs = manager.create(new float[] {6, -9, -12, 15, 0, 4}, new Shape(2, 3));
@@ -446,7 +447,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testMatMul() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // 2D * 2D
             NDArray lhs = manager.create(new float[] {5, 10, -3, 4, 2, 7}, new Shape(2, 3));
             NDArray rhs = manager.create(new float[] {2, 9, 3}, new Shape(3, 1));
@@ -487,7 +488,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testDivScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {6, 9, 12, 15, 0});
             NDArray result = NDArrays.div(dividend, 3);
             NDArray inPlaceResult = NDArrays.divi(dividend, 3);
@@ -503,7 +504,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testDivNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {6, 9, 12, 15, 0});
             NDArray divisor = manager.create(new float[] {2, 3, 4, 5, 6});
             NDArray result = NDArrays.div(dividend, divisor);
@@ -520,7 +521,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseDivScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {6, 9, 12, 15, 45});
             NDArray result = NDArrays.div(180, dividend);
             NDArray inPlaceResult = NDArrays.divi(180, dividend);
@@ -536,7 +537,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseDivNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {6, 9, 12, 15, 45});
             NDArray divisor = manager.create(new float[] {24, 63, 96, 15, 90});
             NDArray result = dividend.getNDArrayInternal().rdiv(divisor);
@@ -566,7 +567,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testModScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {5, 6, 7, 8, 9});
             NDArray result = NDArrays.mod(dividend, 3);
             NDArray inPlaceResult = NDArrays.modi(dividend, 3);
@@ -582,7 +583,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testModNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {7, 8, 9, 10, 11});
             NDArray divisor = manager.create(new float[] {2, 3, 4, 5, 6});
             NDArray result = NDArrays.mod(dividend, divisor);
@@ -599,7 +600,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseModScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {5, 6, 7, 8, 9});
             NDArray result = NDArrays.mod(180, dividend);
             NDArray inPlaceResult = NDArrays.modi(180, dividend);
@@ -617,7 +618,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReverseModNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray dividend = manager.create(new float[] {7, 8, 9, 10, 11});
             NDArray divisor = manager.create(new float[] {20, 21, 22, 23, 24});
             NDArray result = dividend.getNDArrayInternal().rmod(divisor);
@@ -646,7 +647,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testPowScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {6, 0, -1, 5, 2}, new Shape(1, 5));
             NDArray result = NDArrays.pow(array, 2);
             NDArray inPlaceResult = NDArrays.powi(array, 2);
@@ -661,7 +662,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testPowNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {6, 9, 12, 2, 0});
             NDArray power = manager.create(new float[] {3, 0, 1, -2, 3});
             NDArray result = array.pow(power);
@@ -678,7 +679,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testReversePowScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {3, 4, 5, 6, 7});
             NDArray power = NDArrays.pow(2, array);
             NDArray inPlaceResult = NDArrays.powi(2, array);
@@ -696,7 +697,7 @@ public class NDArrayElementArithmeticOpTest {
 
     @Test
     public void testBatchDot() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.ones(new Shape(2, 1, 4));
             NDArray array2 = manager.ones(new Shape(2, 4, 6));
             NDArray expected = manager.create(4f).tile(12).reshape(new Shape(2, 1, 6));
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementComparisonOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementComparisonOpTest.java
index 6842cc5de..7d4da81c7 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementComparisonOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayElementComparisonOpTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDManager;
@@ -25,7 +26,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testContentEquals() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f});
             NDArray array2 = manager.create(new float[] {1f, 2f});
             Assert.assertTrue(NDArrays.shapeEquals(array1, array2));
@@ -71,11 +72,11 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testEqualsForScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray result = NDArrays.eq(array1, 2);
             NDArray expected = manager.create(new boolean[] {false, true, false, false});
-            Assert.assertEquals(result, expected, "Incorrect comparison for equal NDArray");
+            Assert.assertEquals(result, expected);
             array1 = manager.ones(new Shape(4, 5, 2));
             result = NDArrays.eq(array1, 1);
             expected = manager.ones(new Shape(4, 5, 2)).toType(DataType.BOOLEAN, false);
@@ -104,7 +105,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testEqualsForEqualNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray array2 = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray result = NDArrays.eq(array1, array2);
@@ -134,7 +135,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testAllClose() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 0f, 2f, 2f, 4f});
             NDArray array2 = manager.create(new float[] {1.001f, 0.001f, 1.999f, 1.999f, 3.999f});
             Assert.assertTrue(array1.allClose(array2, 0, 0.01, false));
@@ -178,7 +179,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testGreaterThanScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 0f, 2f, 2f, 4f});
             NDArray result = NDArrays.gt(array, 2);
             NDArray expected = manager.create(new boolean[] {false, false, false, false, true});
@@ -211,7 +212,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testGreaterThanNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 2f, 4f, 5f, 4f});
             NDArray array2 = manager.create(new float[] {2f, 1f, 2f, 5f, 4f, 5f});
             NDArray result = NDArrays.gt(array1, array2);
@@ -251,7 +252,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testGreaterThanOrEqualToScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 2f, 4f});
             NDArray result = NDArrays.gte(array, 2);
             NDArray expected = manager.create(new boolean[] {false, true, true, true});
@@ -280,7 +281,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testGreaterThanOrEqualToNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 2f, 4f, 5f, 4f});
             NDArray array2 = manager.create(new float[] {2f, 1f, 2f, 5f, 4f, 5f});
             NDArray result = NDArrays.gte(array1, array2);
@@ -321,7 +322,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testLesserThanScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 2f, 4f, 5f});
             NDArray result = NDArrays.lt(array, 2);
             NDArray expected = manager.create(new boolean[] {true, false, false, false, false});
@@ -350,7 +351,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testLesserThanNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 2f, 4f, 5f});
             NDArray array2 = manager.create(new float[] {2f, 1f, 1f, 5f, 4f});
             NDArray result = NDArrays.lt(array1, array2);
@@ -398,7 +399,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testLesserThanOrEqualToScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10.0f);
             NDArray result = NDArrays.lte(array, 4);
             NDArray expected =
@@ -429,7 +430,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testLesserThanOrEqualToNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.arange(10.0f);
             NDArray array2 = manager.ones(new Shape(10)).mul(5);
             NDArray result = NDArrays.lte(array1, array2);
@@ -468,7 +469,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testMaximumScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10.0f);
             NDArray result = NDArrays.maximum(4f, array);
             NDArray expected = manager.create(new float[] {4f, 4f, 4f, 4f, 4f, 5f, 6f, 7f, 8f, 9f});
@@ -495,7 +496,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testMaximumNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 3f, 4f, 5f});
             NDArray array2 = manager.create(new float[] {5f, 4f, 3f, 2f, 1f});
             NDArray result = NDArrays.maximum(array1, array2);
@@ -535,7 +536,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testMinimumScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10.0f);
             NDArray result = NDArrays.minimum(4f, array);
             NDArray expected = manager.create(new float[] {0f, 1f, 2f, 3f, 4f, 4f, 4f, 4f, 4f, 4f});
@@ -562,7 +563,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testMinimumNDArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 3f, 4f, 5f});
             NDArray array2 = manager.create(new float[] {5f, 4f, 3f, 2f, 1f});
             NDArray result = NDArrays.minimum(array1, array2);
@@ -602,7 +603,7 @@ public class NDArrayElementComparisonOpTest {
 
     @Test
     public void testWhere() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 2f, 4f, 5f, 4f});
             NDArray array2 = manager.create(new float[] {2f, 1f, 3f, 5f, 4f, 5f});
             NDArray condition =
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayLogicalOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayLogicalOpTest.java
index 1224a5af4..19001292c 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayLogicalOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayLogicalOpTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDManager;
@@ -26,7 +27,7 @@ public class NDArrayLogicalOpTest {
 
     @Test
     public void testLogicalAnd() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new boolean[] {true, false});
             NDArray array2 = manager.create(new boolean[] {false, false});
             NDArray expected = manager.create(new boolean[] {false, false});
@@ -80,7 +81,7 @@ public class NDArrayLogicalOpTest {
 
     @Test
     public void testLogicalOr() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new boolean[] {true, false, true, false});
             NDArray array2 = manager.create(new boolean[] {false, true, false, true});
             NDArray expected = manager.create(new boolean[] {true, true, true, true});
@@ -128,7 +129,7 @@ public class NDArrayLogicalOpTest {
 
     @Test
     public void testLogicalXor() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new boolean[] {true, false, true, false});
             NDArray array2 = manager.create(new boolean[] {true, true, false, false});
             NDArray expected = manager.create(new boolean[] {false, true, true, false});
@@ -168,7 +169,7 @@ public class NDArrayLogicalOpTest {
 
     @Test
     public void testLogicalNot() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {-2f, 0f, 1f});
             NDArray expected = manager.create(new boolean[] {false, true, false});
             Assertions.assertAlmostEquals(array.logicalNot(), expected);
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayNumericOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayNumericOpTest.java
index d3516bc08..410b4009a 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayNumericOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayNumericOpTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.Shape;
@@ -26,7 +27,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testNegation() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {6, 9, -12, -11, 0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(x -> -x).toArray();
@@ -55,7 +56,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAbs() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, -2.12312, -3.5784, -4.0, 5.0, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::abs).toArray();
@@ -79,7 +80,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testSquare() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, -2.12312, -3.5784, -4.0, 5.0, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(x -> Math.pow(x, 2.0)).toArray();
@@ -90,7 +91,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testSqrt() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {2.0, 3.0, 4.0, 5.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::sqrt).toArray();
@@ -114,7 +115,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testCbrt() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, -3.584, -4.343234, 5.11111, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::cbrt).toArray();
@@ -138,7 +139,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testFloor() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::floor).toArray();
@@ -162,7 +163,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testCeil() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, -3.584, -4.343234, 5.11111, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::ceil).toArray();
@@ -186,7 +187,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testRound() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, -3.584, -4.343234, 5.11111, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::round).toArray();
@@ -196,9 +197,10 @@ public class NDArrayNumericOpTest {
             data = new double[] {-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0, 2.3};
             array = manager.create(data, new Shape(4, 2));
             // the result of round in Maths is different from Numpy
-            data = new double[] {-2.0, -2.0, 0.0, 0.0, 2.0, 2.0, 2.0, 2.0};
+            data = new double[] {-2.0, -2.0, -0.0, 0.0, 2.0, 2.0, 2.0, 2.0};
             expected = manager.create(data, new Shape(4, 2));
-            Assert.assertEquals(array.round(), expected);
+            NDArray result = array.round();
+            Assert.assertEquals(result, expected);
             // test scalar
             array = manager.create(1.0001f);
             expected = manager.create(1f);
@@ -211,7 +213,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testTrunc() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, -3.584, -4.343234, 5.11111, -223.23423};
             NDArray array = manager.create(data);
             data = new double[] {1.0, 2.0, -3, -4, 5, -223};
@@ -235,7 +237,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testExp() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, -3.584, -4.343234, 5.11111, -223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::exp).toArray();
@@ -259,7 +261,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testGammaln() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {0.5, 1., 1.3782, 1.5};
             NDArray array = manager.create(data);
             data = new double[] {0.5724, 0.0000, -0.1180, -0.1208};
@@ -283,7 +285,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testLog() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::log).toArray();
@@ -307,7 +309,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testLog10() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.12, 3.584, 4.334, 5.111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::log10).toArray();
@@ -331,7 +333,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testLog2() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(x -> Math.log10(x) / Math.log10(2)).toArray();
@@ -355,7 +357,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testSin() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::sin).toArray();
@@ -379,7 +381,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testCos() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::cos).toArray();
@@ -403,7 +405,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testTan() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {0.0, Math.PI / 4.0, Math.PI / 2.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::tan).toArray();
@@ -427,7 +429,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAsin() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, -1.0, -0.22, 0.4, 0.1234};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::asin).toArray();
@@ -451,7 +453,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAcos() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {-1.0, -0.707, 0.0, 0.707, 1.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::acos).toArray();
@@ -475,7 +477,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAtan() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {-1.0, 0.0, 1.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::atan).toArray();
@@ -499,7 +501,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testToDegrees() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {0, Math.PI / 2, Math.PI, 3 * Math.PI / 2, 2 * Math.PI};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::toDegrees).toArray();
@@ -526,7 +528,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testToRadians() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {0.0, 90.0, 180.0, 270.0, 360.0};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::toRadians).toArray();
@@ -554,7 +556,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testSinh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::sinh).toArray();
@@ -578,7 +580,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testCosh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::cosh).toArray();
@@ -602,7 +604,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testTanh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {1.0, 2.2312, 3.584, 4.343234, 5.11111, 223.23423};
             NDArray array = manager.create(data);
             data = DoubleStream.of(data).map(Math::tanh).toArray();
@@ -626,7 +628,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAsinh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {Math.E, 10.0};
             NDArray array = manager.create(data);
             data = new double[] {1.72538256, 2.99822295};
@@ -654,7 +656,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAcosh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {Math.E, 10.0};
             NDArray array = manager.create(data);
             data = new double[] {1.65745445, 2.99322285};
@@ -682,7 +684,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testAtanh() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {0.0, -0.5};
             NDArray array = manager.create(data);
             data = new double[] {0.0, -0.54930614};
@@ -705,7 +707,7 @@ public class NDArrayNumericOpTest {
 
     @Test
     public void testSign() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             double[] data = {-3, 5, -7};
             NDArray array = manager.create(data);
             data = new double[] {-1, 1, -1};
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayOtherOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayOtherOpTest.java
index 663044695..6788a405f 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayOtherOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayOtherOpTest.java
@@ -14,6 +14,7 @@ package ai.djl.integration.tests.ndarray;
 
 import ai.djl.engine.Engine;
 import ai.djl.engine.EngineException;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.LazyNDArray;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
@@ -34,7 +35,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testCopyTo() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray array2 = manager.create(new Shape(4));
             array1.copyTo(array2);
@@ -59,7 +60,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testNonZero() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.ones(new Shape(3, 3));
             NDArray expected =
                     manager.create(
@@ -84,7 +85,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testAll() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10);
             Assert.assertFalse(array.all().getBoolean());
             Assert.assertTrue(array.add(1f).all().getBoolean());
@@ -114,7 +115,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testAny() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10);
             Assert.assertTrue(array.any().getBoolean());
             array = manager.zeros(new Shape(10));
@@ -144,7 +145,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testNone() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10);
             Assert.assertFalse(array.none().getBoolean());
             array = manager.zeros(new Shape(10));
@@ -175,7 +176,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testCountNonzero() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(4);
             Assert.assertEquals(array.countNonzero().getLong(), 3);
 
@@ -194,7 +195,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testIsNaN() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {Float.NaN, 0f});
             NDArray expected = manager.create(new boolean[] {true, false});
             Assert.assertEquals(array.isNaN(), expected);
@@ -222,18 +223,19 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testIntern() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.ones(new Shape(3, 2));
             NDArray replaced = manager.zeros(new Shape(5, 5));
             original.intern(replaced);
             Assert.assertEquals(original, manager.zeros(new Shape(5, 5)));
+            // replace is closed
             Assert.assertThrows(IllegalStateException.class, replaced::toFloatArray);
         }
     }
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testBooleanMask() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(4.0f);
             NDArray index = manager.create(new boolean[] {true, false, true, false});
             NDArray expected = manager.create(new float[] {0f, 2f});
@@ -263,7 +265,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testSet() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test float
             NDArray array = manager.create(1f);
             float[] data = {-1};
@@ -300,7 +302,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testSetBoolean() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(0f, 6f).reshape(2, 3);
             array.set(array.gte(2), 10f);
             NDArray expected = manager.create(new float[] {0, 1, 10, 10, 10, 10}, new Shape(2, 3));
@@ -312,7 +314,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testSequenceMask() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[][] {{1, 2, 3}, {4, 5, 6}});
             NDArray sequenceLength = manager.create(new float[] {1, 2});
             NDArray expected = manager.create(new float[][] {{1, 0, 0}, {4, 5, 0}});
@@ -332,7 +334,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testSequenceMaskWithScalarInput() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new Shape());
             NDArray sequenceLength = manager.create(new float[] {1});
             array.sequenceMask(sequenceLength);
@@ -341,7 +343,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testSequenceMaskWithScalarSequenceLength() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new Shape(1, 1, 1));
             NDArray sequenceLength = manager.create(new float[] {});
             array.sequenceMask(sequenceLength);
@@ -351,7 +353,7 @@ public class NDArrayOtherOpTest {
     @Test
     public void testArgSort() {
         // TODO switch to numpy argsort
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {-1f, 2f, 0f, 999f, -998f});
             NDArray expected = manager.create(new long[] {4, 0, 2, 1, 3});
             Assert.assertEquals(array.argSort(), expected);
@@ -377,7 +379,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = EngineException.class)
     public void testSort() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {2f, 1f, 4f, 3f});
             NDArray expected = manager.create(new float[] {1f, 2f, 3f, 4f});
             Assert.assertEquals(array.sort(), expected);
@@ -427,7 +429,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testSoftmax() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.ones(new Shape(10));
             NDArray expected = manager.zeros(new Shape(10)).add(0.1f);
             Assertions.assertAlmostEquals(array.softmax(0), expected);
@@ -452,7 +454,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testLogSoftmax() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.ones(new Shape(10));
             NDArray expected = manager.zeros(new Shape(10)).add(-2.3025851f);
             Assertions.assertAlmostEquals(array.logSoftmax(0), expected);
@@ -473,7 +475,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testCumsum() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(10.0f);
             NDArray expected =
                     manager.create(new float[] {0f, 1f, 3f, 6f, 10f, 15f, 21f, 28f, 36f, 45f});
@@ -526,7 +528,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testTile() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(2, 2));
 
             NDArray tileAll = array.tile(2);
@@ -570,7 +572,7 @@ public class NDArrayOtherOpTest {
     @Test
     public void testRepeat() {
         // TODO add scalar and zero-dim test cases after fix the bug in MXNet np.repeat
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(2, 2));
 
             NDArray repeatAll = array.repeat(2);
@@ -597,7 +599,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testClip() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray expected = manager.create(new float[] {2f, 2f, 3f, 3f});
             Assert.assertEquals(original.clip(2.0, 3.0), expected);
@@ -627,7 +629,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testSwapAxes() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array =
                     manager.create(new float[] {0f, 1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f})
                             .reshape(new Shape(2, 5));
@@ -649,7 +651,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testFlip() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test single dim
             NDArray original = manager.create(new float[] {1, 2, 3, 4});
             NDArray expected = manager.create(new float[] {4, 3, 2, 1});
@@ -666,7 +668,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testTranspose() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(1, 2, 2));
 
             NDArray transposeAll = original.transpose();
@@ -695,7 +697,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testRot90() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2));
             NDArray rotated = original.rotate90(1, new int[] {0, 1});
             NDArray expected = manager.create(new float[] {2, 4, 1, 3}, new Shape(2, 2));
@@ -717,7 +719,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testBroadcast() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1, 2});
             NDArray broadcasted = array.broadcast(2, 2);
             NDArray expected = manager.create(new float[] {1, 2, 1, 2}, new Shape(2, 2));
@@ -743,7 +745,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = {IllegalArgumentException.class, EngineException.class})
     public void testArgMax() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array =
                     manager.create(
                             new float[] {
@@ -781,7 +783,7 @@ public class NDArrayOtherOpTest {
 
     @Test(expectedExceptions = {IllegalArgumentException.class, EngineException.class})
     public void testArgMin() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array =
                     manager.create(
                             new float[] {
@@ -819,7 +821,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testEncodeDecode() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new byte[] {0, 3}, new Shape(2));
             byte[] bytes = array.encode();
             NDArray recovered = NDArray.decode(manager, bytes);
@@ -841,7 +843,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testErfinv() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test 1-D
             NDArray array = manager.create(new float[] {0f, 0.5f, -1f});
             NDArray expected = manager.create(new float[] {0f, 0.4769f, Float.NEGATIVE_INFINITY});
@@ -875,7 +877,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testInverse() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array =
                     manager.create(
                                     new float[] {
@@ -895,7 +897,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testNorm() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test 1-D
             NDArray array = manager.create(new float[] {1f, 0.5f, -1f});
             Assert.assertEquals(array.norm(), manager.create(1.5f));
@@ -925,7 +927,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testOneHot() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test basic
             NDArray array = manager.create(new int[] {1, 0, 2, 0});
             NDArray expected =
@@ -952,7 +954,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testNormalize() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             float[][] buf = {
                 {0.2673f, 0.5345f, 0.8018f},
                 {0.4558f, 0.5698f, 0.6838f}
@@ -967,11 +969,12 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testStopGradient() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // normal gradient
             NDArray x = manager.create(new float[] {1.0f}, new Shape(1));
             x.setRequiresGradient(true);
-            try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
+            Engine engine = Engine.getEngine(TestUtils.getEngine());
+            try (GradientCollector gc = engine.newGradientCollector()) {
                 NDArray y = x.mul(x);
                 gc.backward(y);
                 NDArray grad = x.getGradient();
@@ -980,7 +983,7 @@ public class NDArrayOtherOpTest {
             // stop gradient
             x = manager.create(new float[] {1.0f}, new Shape(1));
             x.setRequiresGradient(true);
-            try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
+            try (GradientCollector gc = engine.newGradientCollector()) {
                 NDArray z = x.mul(x.stopGradient());
                 gc.backward(z);
                 NDArray grad = x.getGradient();
@@ -991,7 +994,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testFft() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray ele = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray result = ele.fft(5);
             result = result.real().flatten();
@@ -1015,7 +1018,7 @@ public class NDArrayOtherOpTest {
 
     @Test
     public void testStft() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray audio = manager.ones(new Shape(20));
             NDArray window = manager.hanningWindow(20);
             int nfft = 20;
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayReductionOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayReductionOpTest.java
index 4e2d3d501..460aaf802 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayReductionOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayReductionOpTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.DataType;
@@ -24,7 +25,7 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testMax() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 5f, 1f});
             Assert.assertEquals(array.max().getFloat(), 5f);
 
@@ -53,7 +54,7 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testMin() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {2f, 1f, 5f, 0f});
             Assert.assertEquals(array.min().getFloat(), 0f);
 
@@ -82,7 +83,7 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testSum() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 5f});
             Assert.assertEquals(array.sum().getFloat(), 11f);
 
@@ -109,22 +110,25 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testCumProd() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new int[] {1, 2, 3, 4});
             NDArray expected = manager.create(new long[] {1, 2, 6, 24});
-            Assert.assertEquals(array.cumProd(0), expected);
+            NDArray result = array.cumProd(0);
+            Assert.assertEquals(result, expected);
 
+            result = array.cumProd(0, array.getDataType());
             NDArray expectedInt = manager.create(new int[] {1, 2, 6, 24});
-            Assert.assertEquals(array.cumProd(0, array.getDataType()), expectedInt);
+            Assert.assertEquals(result, expectedInt);
 
+            result = array.cumProd(0, DataType.FLOAT32);
             NDArray expectedFloat = manager.create(new float[] {1f, 2f, 6f, 24f});
-            Assert.assertEquals(array.cumProd(0, DataType.FLOAT32), expectedFloat);
+            Assert.assertEquals(result, expectedFloat);
         }
     }
 
     @Test
     public void testProd() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f});
             Assert.assertEquals(array.prod().getFloat(), 24f);
 
@@ -153,7 +157,7 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testMean() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f});
             Assert.assertEquals(array.mean().getFloat(), 2.5f);
 
@@ -180,7 +184,7 @@ public class NDArrayReductionOpTest {
 
     @Test
     public void testTrace() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.arange(8.0f).reshape(new Shape(2, 2, 2)).trace();
             NDArray expect = manager.create(new float[] {6f, 8f});
             Assert.assertEquals(original, expect);
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayShapesManipulationOpTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayShapesManipulationOpTest.java
index dae21cb79..cc756e6c4 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayShapesManipulationOpTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDArrayShapesManipulationOpTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDList;
@@ -27,7 +28,7 @@ public class NDArrayShapesManipulationOpTest {
     @Test
     public void testSplit() {
         // TODO add more test cases once MXNet split op bug is fixed
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.arange(18f);
             NDList result = array.split(18);
             Assert.assertEquals(result.get(0), manager.create(new float[] {0f}));
@@ -51,7 +52,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test
     public void testFlatten() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f});
             NDArray expected = manager.create(new float[] {1f, 2f, 3f, 4f});
             Assert.assertEquals(array.flatten(), expected);
@@ -83,7 +84,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test
     public void testReshape() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f, 3f, 4f, 5f, 6f});
             NDArray expected =
                     manager.create(new float[] {1f, 2f, 3f, 4f, 5f, 6f}, new Shape(2, 1, 1, 3));
@@ -127,7 +128,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test
     public void testExpandDim() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.create(new float[] {1f, 2f});
             NDArray expected = manager.create(new float[] {1f, 2f}, new Shape(1, 2));
             Assert.assertEquals(array.expandDims(0), expected);
@@ -151,7 +152,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test
     public void testSqueeze() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array = manager.ones(new Shape(1, 2, 1, 3, 1));
             NDArray expected = manager.ones(new Shape(2, 3));
             Assert.assertEquals(array.squeeze(), expected);
@@ -179,7 +180,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test
     public void testStack() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Assert.assertThrows(() -> NDArrays.stack(new NDList()));
 
             NDArray array1 = manager.create(new float[] {1f, 2f});
@@ -238,7 +239,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testConcat() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(new float[] {1f});
             NDArray array2 = manager.create(new float[] {2f});
 
@@ -281,7 +282,7 @@ public class NDArrayShapesManipulationOpTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testConcatNDlist() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray array1 = manager.create(1f);
             NDArray array2 = manager.create(2f);
             NDArrays.concat(new NDList(array1, array2));
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDIndexTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDIndexTest.java
index 9e9b7753a..b77cc8f6d 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDIndexTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDIndexTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.index.NDIndex;
@@ -25,7 +26,7 @@ public class NDIndexTest {
 
     @Test
     public void testEmptyIndex() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(2, 2));
             Assert.assertEquals(original.get(new NDIndex()), original);
         }
@@ -33,7 +34,7 @@ public class NDIndexTest {
 
     @Test
     public void testFixedNegativeIndex() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(4));
             NDArray expected = manager.create(4f);
             NDArray actual = original.get("-1");
@@ -43,7 +44,7 @@ public class NDIndexTest {
 
     @Test
     public void testPick() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(2, 2));
             NDArray expected = manager.create(new float[] {1f, 4f}, new Shape(2, 1));
             NDArray actual =
@@ -64,7 +65,7 @@ public class NDIndexTest {
 
     @Test
     public void testGather() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.arange(20f).reshape(-1, 4);
             NDArray index = manager.create(new float[] {0, 0, 2, 1, 1, 2}, new Shape(3, 2));
             NDArray actual = original.gather(index, 1);
@@ -75,7 +76,7 @@ public class NDIndexTest {
 
     @Test
     public void testGatherNd() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.arange(1f, 9f).reshape(2, 2, 2);
             NDArray index = manager.create(new float[] {0, 1, 1, 0}, new Shape(2, 2));
             NDArray actual = original.gatherNd(index);
@@ -95,7 +96,7 @@ public class NDIndexTest {
 
     @Test
     public void testTake() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.arange(1, 7f).reshape(-1, 3);
             NDArray index = manager.create(new float[] {0, 4, 1, 2}, new Shape(2, 2));
             NDArray actual = original.take(index);
@@ -106,7 +107,7 @@ public class NDIndexTest {
 
     @Test
     public void testGet() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1f, 2f, 3f, 4f}, new Shape(2, 2));
             Assert.assertEquals(original.get(new NDIndex()), original);
 
@@ -181,7 +182,7 @@ public class NDIndexTest {
     public void testEmptyArrayClosing() {
         // This is to check the resource closing issue in MXNet engine is circumvented.
         // MXNetError: Check failed: delay_alloc:
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             manager.create(new Shape(2, 2)).get(":4, 3:4");
             manager.create(new Shape(2)).get("3:3");
             manager.create(new Shape(2)).get("2:3");
@@ -190,7 +191,7 @@ public class NDIndexTest {
 
     @Test
     public void testSetArray() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2));
             NDArray expected = manager.create(new float[] {9, 10, 3, 4}, new Shape(2, 2));
             NDArray value = manager.create(new float[] {9, 10});
@@ -214,7 +215,7 @@ public class NDIndexTest {
 
     @Test
     public void testSetArrayBroadcast() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2, 1));
             NDArray expected = manager.create(new float[] {9, 9, 3, 4}, new Shape(2, 2, 1));
             NDArray value = manager.create(new float[] {9});
@@ -225,7 +226,7 @@ public class NDIndexTest {
 
     @Test
     public void testSetNumber() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2));
             NDArray expected = manager.create(new float[] {9, 9, 3, 4}, new Shape(2, 2));
             original.set(new NDIndex(0), 9);
@@ -260,7 +261,7 @@ public class NDIndexTest {
 
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testSetScalar() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2));
             NDArray expected = manager.create(new float[] {0, 2, 3, 4}, new Shape(2, 2));
             original.setScalar(new NDIndex(0, 0), 0);
@@ -271,7 +272,7 @@ public class NDIndexTest {
 
     @Test
     public void testSetByFunction() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.arange(1, 10).reshape(3, 3);
             NDArray expected = manager.create(new int[] {4, 10, 16});
             NDIndex index = new NDIndex(":, 1");
@@ -288,7 +289,7 @@ public class NDIndexTest {
 
     @Test
     public void testSetByFunctionIncrements() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.ones(new Shape(1, 5));
             original.set(new NDIndex(":, 0::2"), array -> array.mul(-1).add(1));
             NDArray expected = manager.create(new float[][] {{0, 1, 0, 1, 0}});
@@ -298,7 +299,7 @@ public class NDIndexTest {
 
     @Test
     public void testPut() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray original = manager.create(new float[] {1, 2, 3, 4}, new Shape(2, 2));
             NDArray expected = manager.create(new float[] {1, 8, 666, 77}, new Shape(2, 2));
             NDArray idx = manager.create(new float[] {2, 3, 1}, new Shape(3));
diff --git a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDListTest.java b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDListTest.java
index 704165a98..ab728c4ea 100644
--- a/integration/src/main/java/ai/djl/integration/tests/ndarray/NDListTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/ndarray/NDListTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.ndarray;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
 
@@ -22,7 +23,7 @@ public class NDListTest {
 
     @Test
     public void testEncode() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Assert.assertThrows(
                     () -> {
                         // invalid NDList length
diff --git a/integration/src/main/java/ai/djl/integration/tests/nn/BlockCoreTest.java b/integration/src/main/java/ai/djl/integration/tests/nn/BlockCoreTest.java
index d166c71b0..084883160 100644
--- a/integration/src/main/java/ai/djl/integration/tests/nn/BlockCoreTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/nn/BlockCoreTest.java
@@ -45,7 +45,6 @@ import ai.djl.nn.recurrent.GRU;
 import ai.djl.nn.recurrent.LSTM;
 import ai.djl.nn.recurrent.RNN;
 import ai.djl.testing.Assertions;
-import ai.djl.testing.TestRequirements;
 import ai.djl.training.DefaultTrainingConfig;
 import ai.djl.training.GradientCollector;
 import ai.djl.training.Trainer;
@@ -76,7 +75,7 @@ public class BlockCoreTest {
 
         long outSize = 3;
         Block block = Linear.builder().setUnits(outSize).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -96,7 +95,7 @@ public class BlockCoreTest {
         }
 
         block = Linear.builder().setUnits(outSize).optBias(false).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -115,7 +114,7 @@ public class BlockCoreTest {
 
         outSize = 10;
         block = Linear.builder().setUnits(outSize).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -139,7 +138,7 @@ public class BlockCoreTest {
 
         long outSize = 3;
         Block block = Linear.builder().setUnits(outSize).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -162,7 +161,7 @@ public class BlockCoreTest {
         }
 
         block = Linear.builder().setUnits(outSize).optBias(false).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -206,10 +205,10 @@ public class BlockCoreTest {
         NDArray expectedLabelsForLinearCollection;
         NDArray dataForLinearCollection;
 
-        try (NDManager sharedManager = NDManager.newBaseManager()) {
+        try (NDManager sharedManager = NDManager.newBaseManager(TestUtils.getEngine())) {
 
             // test expected output when using LinearCollection
-            try (Model model = Model.newInstance("model")) {
+            try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
                 Block block = LinearCollection.builder().setUnits(outSize).build();
                 model.setBlock(block);
 
@@ -248,7 +247,7 @@ public class BlockCoreTest {
 
             // test expected output can be reproduced by using splitSize-many Linear blocks
             for (int splitIndex : IntStream.range(0, splitSize).toArray()) {
-                try (Model model = Model.newInstance("model")) {
+                try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
                     Block block = Linear.builder().setUnits(outSize).build();
                     model.setBlock(block);
 
@@ -292,7 +291,7 @@ public class BlockCoreTest {
             }
 
             // test expected output when using multiple split axis
-            try (Model model = Model.newInstance("model")) {
+            try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
                 Block block = LinearCollection.builder().setUnits(outSize).build();
                 model.setBlock(block);
 
@@ -346,12 +345,12 @@ public class BlockCoreTest {
         // store sum on Multiplication block's result
         NDArray sum;
 
-        try (NDManager sharedManager = NDManager.newBaseManager()) {
+        try (NDManager sharedManager = NDManager.newBaseManager(TestUtils.getEngine())) {
 
             // test algebraic expectation of Multiplication block
             long outSize = 2;
             Block block = Multiplication.builder().setUnits(outSize).build();
-            try (Model model = Model.newInstance("model")) {
+            try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
                 model.setBlock(block);
 
                 TrainingConfig config =
@@ -378,7 +377,7 @@ public class BlockCoreTest {
 
             // test "sum" is equal to linear transformation without bias
             block = Linear.builder().setUnits(outSize).optBias(false).build();
-            try (Model model = Model.newInstance("model")) {
+            try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
                 model.setBlock(block);
 
                 TrainingConfig config =
@@ -408,7 +407,7 @@ public class BlockCoreTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = BatchNorm.builder().build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -436,7 +435,7 @@ public class BlockCoreTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = GhostBatchNorm.builder().build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -466,14 +465,14 @@ public class BlockCoreTest {
     @SuppressWarnings("try")
     @Test
     public void testLayerNorm() throws IOException, MalformedModelException {
-        TestRequirements.engine("PyTorch", "MXNet");
+        TestUtils.requiresEngine("PyTorch", "MXNet");
 
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = LayerNorm.builder().build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -495,14 +494,14 @@ public class BlockCoreTest {
     @SuppressWarnings("try")
     @Test
     public void test2LayerNorm() throws IOException, MalformedModelException {
-        TestRequirements.engine("PyTorch", "MXNet");
+        TestUtils.requiresEngine("PyTorch", "MXNet");
 
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = LayerNorm.builder().axis(2, 3).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -529,7 +528,7 @@ public class BlockCoreTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = Dropout.builder().optRate(.5f).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -564,7 +563,7 @@ public class BlockCoreTest {
                                         .build())
                         .setEmbeddingSize(2)
                         .build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -596,7 +595,7 @@ public class BlockCoreTest {
         Block block =
                 Conv1d.builder().setKernelShape(new Shape(2)).setFilters(1).optBias(false).build();
 
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -633,7 +632,7 @@ public class BlockCoreTest {
                         .optBias(false)
                         .build();
 
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -662,7 +661,7 @@ public class BlockCoreTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = Conv2d.builder().setKernelShape(new Shape(2, 2)).setFilters(1).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -695,7 +694,7 @@ public class BlockCoreTest {
 
         Block block =
                 Conv2dTranspose.builder().setKernelShape(new Shape(2, 2)).setFilters(1).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -732,7 +731,7 @@ public class BlockCoreTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
         Block block = Conv3d.builder().setKernelShape(new Shape(2, 2, 2)).setFilters(1).build();
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -770,7 +769,7 @@ public class BlockCoreTest {
         TrainingConfig config =
                 new DefaultTrainingConfig(loss)
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
-                        .optDevices(TestUtils.getDevices());
+                        .optDevices(TestUtils.getDevices(Integer.MAX_VALUE));
         Block block =
                 RNN.builder()
                         .setStateSize(4)
@@ -779,14 +778,15 @@ public class BlockCoreTest {
                         .optBatchFirst(true)
                         .optReturnState(true)
                         .build();
-        try (Model model = Model.newInstance("model", config.getDevices()[0])) {
+        try (Model model =
+                Model.newInstance("model", config.getDevices()[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
                 // the unused GradientCollector is for BatchNorm to know it is on training mode
                 try (GradientCollector collector = trainer.newGradientCollector()) {
                     Shape inputShape = new Shape(1, 2, 4);
-                    Engine.getInstance().setRandomSeed(1234);
+                    Engine.getEngine(TestUtils.getEngine()).setRandomSeed(1234);
                     trainer.initialize(inputShape);
                     NDManager manager = trainer.getManager();
                     NDArray data =
@@ -817,7 +817,7 @@ public class BlockCoreTest {
         TrainingConfig config =
                 new DefaultTrainingConfig(loss)
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
-                        .optDevices(TestUtils.getDevices());
+                        .optDevices(TestUtils.getDevices(Integer.MAX_VALUE));
         Block block =
                 RNN.builder()
                         .setStateSize(4)
@@ -826,14 +826,15 @@ public class BlockCoreTest {
                         .optBatchFirst(true)
                         .optReturnState(true)
                         .build();
-        try (Model model = Model.newInstance("model", config.getDevices()[0])) {
+        try (Model model =
+                Model.newInstance("model", config.getDevices()[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
                 // the unused GradientCollector is for BatchNorm to know it is on training mode
                 try (GradientCollector collector = trainer.newGradientCollector()) {
                     Shape inputShape = new Shape(1, 2, 4);
-                    Engine.getInstance().setRandomSeed(1234);
+                    Engine.getEngine(TestUtils.getEngine()).setRandomSeed(1234);
                     trainer.initialize(inputShape);
                     NDManager manager = trainer.getManager();
                     NDArray data =
@@ -867,7 +868,7 @@ public class BlockCoreTest {
         TrainingConfig config =
                 new DefaultTrainingConfig(loss)
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
-                        .optDevices(TestUtils.getDevices());
+                        .optDevices(TestUtils.getDevices(Integer.MAX_VALUE));
         Block block =
                 LSTM.builder()
                         .setStateSize(4)
@@ -875,14 +876,15 @@ public class BlockCoreTest {
                         .optBatchFirst(true)
                         .optReturnState(true)
                         .build();
-        try (Model model = Model.newInstance("model", config.getDevices()[0])) {
+        try (Model model =
+                Model.newInstance("model", config.getDevices()[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
                 // the unused GradientCollector is for BatchNorm to know it is on training mode
                 try (GradientCollector collector = trainer.newGradientCollector()) {
                     Shape inputShape = new Shape(1, 2, 4);
-                    Engine.getInstance().setRandomSeed(1234);
+                    Engine.getEngine(TestUtils.getEngine()).setRandomSeed(1234);
                     trainer.initialize(inputShape);
                     NDManager manager = trainer.getManager();
                     NDArray data =
@@ -917,7 +919,7 @@ public class BlockCoreTest {
         TrainingConfig config =
                 new DefaultTrainingConfig(loss)
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
-                        .optDevices(TestUtils.getDevices());
+                        .optDevices(TestUtils.getDevices(Integer.MAX_VALUE));
         GRU block =
                 GRU.builder()
                         .setStateSize(4)
@@ -925,14 +927,15 @@ public class BlockCoreTest {
                         .optBatchFirst(true)
                         .optReturnState(false)
                         .build();
-        try (Model model = Model.newInstance("model", config.getDevices()[0])) {
+        try (Model model =
+                Model.newInstance("model", config.getDevices()[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
                 // the unused GradientCollector is for BatchNorm to know it is on training mode
                 try (GradientCollector collector = trainer.newGradientCollector()) {
                     Shape inputShape = new Shape(1, 2, 4);
-                    Engine.getInstance().setRandomSeed(1234);
+                    Engine.getEngine(TestUtils.getEngine()).setRandomSeed(1234);
                     trainer.initialize(inputShape);
                     NDManager manager = trainer.getManager();
                     NDArray data =
@@ -990,7 +993,7 @@ public class BlockCoreTest {
         block.removeLastBlock();
         Assert.assertEquals(block.getChildren().size(), 4);
 
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -1027,7 +1030,7 @@ public class BlockCoreTest {
         Assert.assertEquals(block.getDirectParameters().size(), 0);
         Assert.assertEquals(block.getParameters().size(), 4);
 
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/nn/BlockFactoryTest.java b/integration/src/main/java/ai/djl/integration/tests/nn/BlockFactoryTest.java
index 93a0968f8..d7ce5b163 100644
--- a/integration/src/main/java/ai/djl/integration/tests/nn/BlockFactoryTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/nn/BlockFactoryTest.java
@@ -15,8 +15,9 @@ package ai.djl.integration.tests.nn;
 import ai.djl.Application;
 import ai.djl.MalformedModelException;
 import ai.djl.Model;
-import ai.djl.engine.Engine;
+import ai.djl.ModelException;
 import ai.djl.inference.Predictor;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.modality.Classifications;
 import ai.djl.modality.cv.Image;
 import ai.djl.ndarray.NDList;
@@ -49,8 +50,7 @@ public class BlockFactoryTest {
 
     @Test
     public void testBlockFactoryLoadingFromZip()
-            throws MalformedModelException, ModelNotFoundException, IOException,
-                    TranslateException {
+            throws ModelException, IOException, TranslateException {
         Path savedDir = Paths.get("build/testBlockFactory");
         Utils.deleteQuietly(savedDir);
         Path zipPath;
@@ -58,7 +58,7 @@ public class BlockFactoryTest {
             zipPath = prepareModel(savedDir);
         } catch (ModelNotFoundException e) {
             throw new UnsupportedOperationException(
-                    "No test model for engine: " + Engine.getDefaultEngineName(), e);
+                    "No test model for engine: " + TestUtils.getEngine(), e);
         }
         // load model from here
         Criteria<NDList, NDList> criteria =
@@ -66,6 +66,7 @@ public class BlockFactoryTest {
                         .setTypes(NDList.class, NDList.class)
                         .optModelPath(zipPath)
                         .optModelName("exported")
+                        .optEngine(TestUtils.getEngine())
                         .build();
         try (ZooModel<NDList, NDList> model = criteria.loadModel();
                 Predictor<NDList, NDList> pred = model.newPredictor()) {
@@ -75,11 +76,10 @@ public class BlockFactoryTest {
         }
     }
 
-    private Path prepareModel(Path savedDir)
-            throws IOException, ModelNotFoundException, MalformedModelException {
+    private Path prepareModel(Path savedDir) throws IOException, ModelException {
         TestBlockFactory factory = new TestBlockFactory();
         try (Model model = factory.getRemoveLastBlockModel();
-                NDManager manager = NDManager.newBaseManager()) {
+                NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Block block = model.getBlock();
             block.forward(
                     new ParameterStore(manager, true),
@@ -93,7 +93,7 @@ public class BlockFactoryTest {
                 Paths.get(
                         "build/classes/java/main/ai/djl/integration/tests/nn/BlockFactoryTest$TestBlockFactory.class"),
                 classDir.resolve("BlockFactoryTest$TestBlockFactory.class"));
-        Path zipPath = Paths.get("build/testBlockFactory" + Engine.getDefaultEngineName() + ".zip");
+        Path zipPath = Paths.get("build/testBlockFactory" + TestUtils.getEngine() + ".zip");
         Files.deleteIfExists(zipPath);
         ZipUtils.zip(savedDir, zipPath, false);
         return zipPath;
@@ -113,7 +113,7 @@ public class BlockFactoryTest {
 
         public Model getRemoveLastBlockModel()
                 throws MalformedModelException, ModelNotFoundException, IOException {
-            String name = Engine.getDefaultEngineName();
+            String name = TestUtils.getEngine();
             Criteria.Builder<Image, Classifications> builder =
                     Criteria.builder()
                             .optApplication(Application.CV.IMAGE_CLASSIFICATION)
diff --git a/integration/src/main/java/ai/djl/integration/tests/nn/PoolingOperationsTest.java b/integration/src/main/java/ai/djl/integration/tests/nn/PoolingOperationsTest.java
index 4cf5fc72c..1e01d4c6c 100644
--- a/integration/src/main/java/ai/djl/integration/tests/nn/PoolingOperationsTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/nn/PoolingOperationsTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.nn;
 
 import ai.djl.Model;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -30,13 +31,14 @@ import org.testng.Assert;
 import org.testng.annotations.Test;
 
 public class PoolingOperationsTest {
+
     TrainingConfig config =
             new DefaultTrainingConfig(Loss.l2Loss())
                     .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT);
 
     @Test
     public void testMaxPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.maxPool1dBlock(new Shape(2)));
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -55,7 +57,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testMaxPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.maxPool2dBlock(new Shape(2, 2)));
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -74,7 +76,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testMaxPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.maxPool3dBlock(new Shape(2, 2, 2)));
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -93,7 +95,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalMaxPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalMaxPool1dBlock());
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -112,7 +114,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalMaxPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalMaxPool2dBlock());
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -131,7 +133,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalMaxPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalMaxPool3dBlock());
             // Look for a max pool value 5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -150,7 +152,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testAvgPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.avgPool1dBlock(new Shape(2)));
             // Look for a average pool value 1.5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -169,7 +171,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testAvgPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.avgPool2dBlock(new Shape(2, 2)));
             // Look for a average pool value 1.25
             try (Trainer trainer = model.newTrainer(config)) {
@@ -188,7 +190,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testAvgPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.avgPool3dBlock(new Shape(2, 2, 2)));
             // Look for a average pool value 1.125
             try (Trainer trainer = model.newTrainer(config)) {
@@ -207,7 +209,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalAvgPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalAvgPool1dBlock());
             // Look for a average pool value 1.5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -226,7 +228,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalAvgPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalAvgPool2dBlock());
             // Look for a average pool value 1.5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -245,7 +247,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalAvgPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalAvgPool3dBlock());
             // Look for a average pool value 1.5
             try (Trainer trainer = model.newTrainer(config)) {
@@ -264,7 +266,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testLpPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.lpPool1dBlock(1, new Shape(2)));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2));
@@ -283,7 +285,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testLpPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.lpPool2dBlock(1, new Shape(2, 2)));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2, 2));
@@ -303,7 +305,7 @@ public class PoolingOperationsTest {
     // lpPool3d is not supported in PyTorch engine
     @Test(expectedExceptions = IllegalArgumentException.class)
     public void testLpPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.lpPool3dBlock(1, new Shape(2, 2, 2)));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2, 2, 2));
@@ -322,7 +324,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalLpPool1d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalLpPool1dBlock(1));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2));
@@ -341,7 +343,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalLpPool2d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalLpPool2dBlock(1));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2, 2));
@@ -360,7 +362,7 @@ public class PoolingOperationsTest {
 
     @Test
     public void testGlobalLpPool3d() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Pool.globalLpPool3dBlock(1));
             try (Trainer trainer = model.newTrainer(config)) {
                 trainer.initialize(new Shape(2, 2, 2, 2, 2));
diff --git a/integration/src/main/java/ai/djl/integration/tests/nn/ScaledDotProductAttentionBlockTest.java b/integration/src/main/java/ai/djl/integration/tests/nn/ScaledDotProductAttentionBlockTest.java
index 46534ccbf..71c731519 100644
--- a/integration/src/main/java/ai/djl/integration/tests/nn/ScaledDotProductAttentionBlockTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/nn/ScaledDotProductAttentionBlockTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.nn;
 
 import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDList;
@@ -720,7 +721,7 @@ public class ScaledDotProductAttentionBlockTest {
         Shape fromShape = new Shape(batchSize, fromSeqLength, embeddingSize);
         Shape toShape = new Shape(batchSize, toSeqLength, embeddingSize);
 
-        NDManager manager = NDManager.newBaseManager();
+        NDManager manager = NDManager.newBaseManager(TestUtils.getEngine());
         NDArray keySequence =
                 manager.create(keySequenceInitialization)
                         .toType(DataType.FLOAT32, false)
@@ -763,7 +764,8 @@ public class ScaledDotProductAttentionBlockTest {
         ParameterStore ps = new ParameterStore(manager, false);
         NDArray result;
         // the unused GradientCollector is for Dropout to know it is on training mode
-        try (GradientCollector collector = Engine.getInstance().newGradientCollector()) {
+        Engine engine = Engine.getEngine(TestUtils.getEngine());
+        try (GradientCollector collector = engine.newGradientCollector()) {
             result =
                     block.forward(
                                     ps,
@@ -777,6 +779,7 @@ public class ScaledDotProductAttentionBlockTest {
         }
         boolean allClose = NDArrays.allClose(result, expectedResult, 1e-04, 1e-07, true);
         Assert.assertTrue(allClose);
+        manager.close();
     }
 
     public static class TestConstantInitializer implements Initializer {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/ActivationTest.java b/integration/src/main/java/ai/djl/integration/tests/training/ActivationTest.java
index 8ff3aa36f..49916460b 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/ActivationTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/ActivationTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.training;
 
 import ai.djl.Model;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -38,7 +39,7 @@ public class ActivationTest {
 
     @Test
     public void testRelu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.reluBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -57,7 +58,7 @@ public class ActivationTest {
 
     @Test
     public void testSigmoid() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.sigmoidBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -74,7 +75,7 @@ public class ActivationTest {
 
     @Test
     public void testTanh() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.tanhBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -91,7 +92,7 @@ public class ActivationTest {
 
     @Test
     public void testSoftPlus() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.softPlusBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -108,7 +109,7 @@ public class ActivationTest {
 
     @Test
     public void testSoftSign() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.softSignBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -125,7 +126,7 @@ public class ActivationTest {
 
     @Test
     public void testLeakyrelu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             float alpha = 1.0f;
             model.setBlock(Activation.leakyReluBlock(alpha));
 
@@ -143,7 +144,7 @@ public class ActivationTest {
 
     @Test
     public void testElu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             float alpha = 1.0f;
             model.setBlock(Activation.eluBlock(alpha));
 
@@ -160,7 +161,7 @@ public class ActivationTest {
 
     @Test
     public void testSelu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.seluBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -177,7 +178,7 @@ public class ActivationTest {
 
     @Test
     public void testGelu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.geluBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -194,7 +195,7 @@ public class ActivationTest {
 
     @Test
     public void testSwish() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             float beta = 1.0f;
             model.setBlock(Activation.swishBlock(beta));
 
@@ -212,7 +213,7 @@ public class ActivationTest {
 
     @Test
     public void testMish() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.mishBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -229,7 +230,7 @@ public class ActivationTest {
 
     @Test
     public void testPrelu() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Activation.preluBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -245,7 +246,7 @@ public class ActivationTest {
 
     @Test
     public void testSparseMax() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(new SparseMax());
 
             try (Trainer trainer = model.newTrainer(config)) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/BlocksTest.java b/integration/src/main/java/ai/djl/integration/tests/training/BlocksTest.java
index 01df52766..d335beff1 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/BlocksTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/BlocksTest.java
@@ -13,6 +13,7 @@
 package ai.djl.integration.tests.training;
 
 import ai.djl.Model;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -37,7 +38,7 @@ public class BlocksTest {
 
     @Test
     public void testFlattenBlock() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.batchFlattenBlock());
 
             try (Trainer trainer = model.newTrainer(config)) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/DatasetTest.java b/integration/src/main/java/ai/djl/integration/tests/training/DatasetTest.java
index 878a300af..1fe46026f 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/DatasetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/DatasetTest.java
@@ -14,6 +14,7 @@ package ai.djl.integration.tests.training;
 
 import ai.djl.Model;
 import ai.djl.basicdataset.cv.classification.Cifar10;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.DataType;
@@ -52,7 +53,7 @@ public class DatasetTest {
 
     @Test
     public void testSequenceSampler() throws IOException, TranslateException {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
 
             NDManager manager = model.getNDManager();
@@ -81,7 +82,7 @@ public class DatasetTest {
 
     @Test
     public void testRandomSampler() throws IOException, TranslateException {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
 
             NDManager manager = model.getNDManager();
@@ -108,7 +109,7 @@ public class DatasetTest {
 
     @Test
     public void testBatchSampler() throws IOException, TranslateException {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
 
             NDManager manager = model.getNDManager();
@@ -191,7 +192,7 @@ public class DatasetTest {
 
     @Test
     public void testArrayDataset() throws IOException, TranslateException {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
 
             NDManager manager = model.getNDManager();
@@ -269,7 +270,7 @@ public class DatasetTest {
 
     @Test
     public void testMultithreading() throws IOException, InterruptedException, TranslateException {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             model.setBlock(Blocks.identityBlock());
             NDManager manager = model.getNDManager();
 
@@ -300,14 +301,15 @@ public class DatasetTest {
 
     @Test
     public void testDatasetToArray() throws IOException, TranslateException {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             RandomAccessDataset dataset =
                     new ArrayDataset.Builder()
                             .setData(manager.ones(new Shape(5, 4)))
                             .setSampling(32, false)
                             .optLabels(manager.zeros(new Shape(5, 2)))
                             .build();
-            Pair<Number[][], Number[][]> converted = dataset.toArray();
+            Pair<Number[][], Number[][]> converted = dataset.toArray(manager);
             Number[][] data = converted.getKey();
             Number[][] labels = converted.getValue();
 
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/EvaluateDatasetTest.java b/integration/src/main/java/ai/djl/integration/tests/training/EvaluateDatasetTest.java
index f3d657a24..22adddbfb 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/EvaluateDatasetTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/EvaluateDatasetTest.java
@@ -15,6 +15,7 @@ package ai.djl.integration.tests.training;
 import ai.djl.Model;
 import ai.djl.basicdataset.cv.classification.Mnist;
 import ai.djl.basicmodelzoo.basic.Mlp;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.metric.Metrics;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.Shape;
@@ -39,7 +40,7 @@ public class EvaluateDatasetTest {
 
     @Test
     public void testDatasetEvaluation() throws IOException, TranslateException {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             Mnist testMnistDataset =
                     Mnist.builder()
                             .optManager(manager)
@@ -51,7 +52,7 @@ public class EvaluateDatasetTest {
 
             Mlp mlpModel = new Mlp(784, 1, new int[] {256}, Activation::relu);
 
-            try (Model model = Model.newInstance("lin-reg")) {
+            try (Model model = Model.newInstance("lin-reg", TestUtils.getEngine())) {
                 model.setBlock(mlpModel);
 
                 Loss l2loss = Loss.l2Loss();
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java b/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
index 0135f9cfb..4638cfd03 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/EvaluatorTest.java
@@ -13,6 +13,7 @@
 
 package ai.djl.integration.tests.training;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -29,7 +30,7 @@ public class EvaluatorTest {
 
     @Test
     public void testAccuracy() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
 
             NDArray predictions =
                     manager.create(new float[] {0.3f, 0.7f, 0, 1, 0.4f, 0.6f}, new Shape(3, 2));
@@ -61,7 +62,7 @@ public class EvaluatorTest {
 
     @Test
     public void testCoverage() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
 
             NDArray predictions = manager.create(new float[] {0.3f, 0.7f, 0.0f}, new Shape(3));
             NDArray labels = manager.create(new float[] {0.5f, 0.5f, 0.5f}, new Shape(3));
@@ -80,7 +81,7 @@ public class EvaluatorTest {
 
     @Test
     public void testTopKAccuracy() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray predictions =
                     manager.create(
                             new float[] {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/GradientCollectorIntegrationTest.java b/integration/src/main/java/ai/djl/integration/tests/training/GradientCollectorIntegrationTest.java
index 92f491ba5..2ad4f5e3b 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/GradientCollectorIntegrationTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/GradientCollectorIntegrationTest.java
@@ -15,6 +15,7 @@ package ai.djl.integration.tests.training;
 import ai.djl.Model;
 import ai.djl.basicmodelzoo.basic.Mlp;
 import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDList;
@@ -52,7 +53,7 @@ public class GradientCollectorIntegrationTest {
 
     @Test
     public void testAutograd() {
-        try (Model model = Model.newInstance("model");
+        try (Model model = Model.newInstance("model", TestUtils.getEngine());
                 NDManager manager = model.getNDManager()) {
             model.setBlock(Blocks.identityBlock());
             try (Trainer trainer =
@@ -83,11 +84,12 @@ public class GradientCollectorIntegrationTest {
 
     @Test
     public void testZeroGradients() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray a = manager.create(0.0f);
             a.setRequiresGradient(true);
 
-            try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
+            Engine engine = Engine.getEngine(TestUtils.getEngine());
+            try (GradientCollector gc = engine.newGradientCollector()) {
                 NDArray b = a.mul(2);
 
                 // Gradients are initially zero
@@ -107,12 +109,13 @@ public class GradientCollectorIntegrationTest {
     /** Tests that the gradients do not accumulate when closing the gradient collector. */
     @Test
     public void testClearGradients() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray a = manager.create(0.0f);
             a.setRequiresGradient(true);
 
+            Engine engine = Engine.getEngine(TestUtils.getEngine());
             for (int i = 0; i < 3; i++) {
-                try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
+                try (GradientCollector gc = engine.newGradientCollector()) {
                     NDArray b = a.mul(2);
                     gc.backward(b);
                 }
@@ -126,11 +129,12 @@ public class GradientCollectorIntegrationTest {
     public void testAccumulateGradients() {
         // TODO: MXNet support for accumulating gradients does not currently work
         TestRequirements.notEngine("MXNet");
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray a = manager.create(0.0f);
             a.setRequiresGradient(true);
 
-            try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
+            Engine engine = Engine.getEngine(TestUtils.getEngine());
+            try (GradientCollector gc = engine.newGradientCollector()) {
                 for (int i = 1; i <= 3; i++) {
                     NDArray b = a.mul(2);
                     gc.backward(b);
@@ -150,8 +154,9 @@ public class GradientCollectorIntegrationTest {
         Assert.assertThrows(
                 () -> {
                     GradientCollector gc2 = null;
-                    try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {
-                        gc2 = Engine.getInstance().newGradientCollector();
+                    Engine engine = Engine.getEngine(TestUtils.getEngine());
+                    try (GradientCollector gc = engine.newGradientCollector()) {
+                        gc2 = engine.newGradientCollector();
                         gc2.close();
                     } finally {
                         if (gc2 != null) {
@@ -163,7 +168,7 @@ public class GradientCollectorIntegrationTest {
 
     @Test
     public void testFreezeParameters() {
-        try (Model model = Model.newInstance("model")) {
+        try (Model model = Model.newInstance("model", TestUtils.getEngine())) {
             Block blockFrozen = new Mlp(10, 10, new int[] {10});
             Block blockNormal = new Mlp(10, 10, new int[] {10});
             Block combined = new SequentialBlock().add(blockFrozen).add(blockNormal);
@@ -239,7 +244,7 @@ public class GradientCollectorIntegrationTest {
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optimizer);
 
-        try (Model model = Model.newInstance("linear")) {
+        try (Model model = Model.newInstance("linear", TestUtils.getEngine())) {
             Linear block = Linear.builder().setUnits(1).build();
             model.setBlock(block);
 
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java b/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
index 1fb74cccc..6dbda5ec5 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/LossTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.training;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -26,7 +27,8 @@ public class LossTest {
 
     @Test
     public void l1LossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(5));
             Assert.assertEquals(
@@ -36,7 +38,8 @@ public class LossTest {
 
     @Test
     public void quantileL1LossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(5));
             Assert.assertEquals(
@@ -49,7 +52,8 @@ public class LossTest {
 
     @Test
     public void l2LossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(5));
             Assert.assertEquals(
@@ -59,7 +63,8 @@ public class LossTest {
 
     @Test
     public void softmaxCrossEntropyTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             // test fromLogits=true, sparseLabel=true
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(1));
@@ -90,7 +95,8 @@ public class LossTest {
 
     @Test
     public void hingeLossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(5)).neg();
             Assert.assertEquals(
@@ -101,7 +107,8 @@ public class LossTest {
 
     @Test
     public void sigmoidBinaryCrossEntropyLossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.create(new float[] {1, 2, 3, 4, 5});
             NDArray label = manager.ones(new Shape(5));
             Assertions.assertAlmostEquals(
@@ -114,7 +121,8 @@ public class LossTest {
 
     @Test
     public void maskedSoftmaxCrossEntropyLossTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        TestUtils.requiresEngine("MXNet", "PyTorch", "TensorFlow");
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray pred = manager.ones(new Shape(3, 4, 10));
             NDArray label = manager.ones(new Shape(3, 4));
             NDArray validLengths = manager.create(new int[] {4, 2, 0});
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/ModelTest.java b/integration/src/main/java/ai/djl/integration/tests/training/ModelTest.java
index c3e461292..ca6801290 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/ModelTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/ModelTest.java
@@ -14,6 +14,7 @@ package ai.djl.integration.tests.training;
 
 import ai.djl.MalformedModelException;
 import ai.djl.Model;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.types.DataType;
 import ai.djl.ndarray.types.Shape;
 import ai.djl.nn.ParameterList;
@@ -35,8 +36,8 @@ public class ModelTest {
         SequentialBlock block = new SequentialBlock();
         block.add(Conv2d.builder().setKernelShape(new Shape(1, 1)).setFilters(10).build());
         block.add(BatchNorm.builder().build());
-        try (Model saveModel = Model.newInstance("saveModel");
-                Model loadModel = Model.newInstance("loadModel")) {
+        try (Model saveModel = Model.newInstance("saveModel", TestUtils.getEngine());
+                Model loadModel = Model.newInstance("loadModel", TestUtils.getEngine())) {
             block.initialize(saveModel.getNDManager(), DataType.FLOAT32, new Shape(1, 3, 32, 32));
             ParameterList savedParameters = block.getParameters();
             saveModel.setBlock(block);
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/OptimizerTest.java b/integration/src/main/java/ai/djl/integration/tests/training/OptimizerTest.java
index b54225dfb..9823a14c2 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/OptimizerTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/OptimizerTest.java
@@ -14,7 +14,7 @@ package ai.djl.integration.tests.training;
 
 import ai.djl.Device;
 import ai.djl.Model;
-import ai.djl.engine.Engine;
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDArrays;
 import ai.djl.ndarray.NDList;
@@ -46,14 +46,14 @@ public class OptimizerTest {
     public void testSgd() {
         Optimizer sgd = Optimizer.sgd().setLearningRateTracker(Tracker.fixed(0.1f)).build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(sgd)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -78,14 +78,14 @@ public class OptimizerTest {
                         .optMomentum(0.9f)
                         .build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -118,14 +118,14 @@ public class OptimizerTest {
                         .build();
 
         // Limit to 1 GPU for consist result.
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             int batchSize = config.getDevices().length * BATCH_SIZE;
@@ -147,14 +147,14 @@ public class OptimizerTest {
     public void testAdam() {
         Optimizer optim = Optimizer.adam().optLearningRateTracker(Tracker.fixed(0.1f)).build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -176,14 +176,14 @@ public class OptimizerTest {
     public void testAdamW() {
         Optimizer optim = Optimizer.adamW().optLearningRateTracker(Tracker.fixed(0.1f)).build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -205,14 +205,14 @@ public class OptimizerTest {
     public void testAdagrad() {
         Optimizer optim = Optimizer.adagrad().optLearningRateTracker(Tracker.fixed(0.1f)).build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -238,14 +238,14 @@ public class OptimizerTest {
                         .optCentered(false)
                         .build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -272,14 +272,14 @@ public class OptimizerTest {
                         .optCentered(true)
                         .build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
                         .optOptimizer(optim)
                         .optDevices(devices);
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
@@ -302,7 +302,7 @@ public class OptimizerTest {
     public void testAdadelta() {
         Optimizer optim = Optimizer.adadelta().build();
 
-        Device[] devices = Engine.getInstance().getDevices(1);
+        Device[] devices = TestUtils.getDevices(1);
         TrainingConfig config =
                 new DefaultTrainingConfig(Loss.l2Loss())
                         .optInitializer(Initializer.ONES, Parameter.Type.WEIGHT)
@@ -310,7 +310,7 @@ public class OptimizerTest {
                         .optDevices(devices);
 
         Block block = Linear.builder().setUnits(CHANNELS).build();
-        try (Model model = Model.newInstance("model", devices[0])) {
+        try (Model model = Model.newInstance("model", devices[0], TestUtils.getEngine())) {
             model.setBlock(block);
 
             try (Trainer trainer = model.newTrainer(config)) {
diff --git a/integration/src/main/java/ai/djl/integration/tests/training/WeightDecayTest.java b/integration/src/main/java/ai/djl/integration/tests/training/WeightDecayTest.java
index b0616af91..2efe211ea 100644
--- a/integration/src/main/java/ai/djl/integration/tests/training/WeightDecayTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/training/WeightDecayTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.training;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -27,7 +28,7 @@ public class WeightDecayTest {
 
     @Test
     public void l1DecayTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray parameters1 = manager.create(new float[] {-1, -2, 3, 4, 5}); // 15
             NDArray parameters2 = manager.create(new float[] {-1, -1, -1, -1, -1}); // 5
             // Not used
@@ -43,7 +44,7 @@ public class WeightDecayTest {
 
     @Test
     public void l2DecayTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray parameters1 = manager.create(new float[] {-1, -2, 3, 4, 5}); // 55
             NDArray parameters2 = manager.create(new float[] {-1, -1, -1, -1, -1}); // 5
             // Not used
@@ -59,7 +60,7 @@ public class WeightDecayTest {
 
     @Test
     public void elasticNetDecayTest() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDArray parameters1 = manager.create(new float[] {-1, -2, 3, 4, 5});
             NDArray parameters2 = manager.create(new float[] {-1, -1, -1, -1, -1});
             // Not used
diff --git a/integration/src/main/java/ai/djl/integration/tests/translate/PaddingStackBatchifierTest.java b/integration/src/main/java/ai/djl/integration/tests/translate/PaddingStackBatchifierTest.java
index 64f541681..6e616ff75 100644
--- a/integration/src/main/java/ai/djl/integration/tests/translate/PaddingStackBatchifierTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/translate/PaddingStackBatchifierTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.translate;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDArray;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
@@ -26,7 +27,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testBatchify() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList[] input = new NDList[5];
             for (int i = 0; i < 5; i++) {
                 NDArray array1 = manager.zeros(new Shape(10, i + 1));
@@ -53,7 +54,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testBatchifyWithValidLength() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList[] input = new NDList[5];
             long[] validLengths = new long[5];
             for (int i = 0; i < 5; i++) {
@@ -80,7 +81,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testBatchifyWithPaddingSize() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList[] input = new NDList[5];
             for (int i = 0; i < 5; i++) {
                 input[i] =
@@ -101,7 +102,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testUnbatchify() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input =
                     new NDList(manager.zeros(new Shape(10, 11)), manager.zeros(new Shape(10)));
             Batchifier batchifier =
@@ -122,7 +123,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testUnbatchifyWithValidLengths() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input = new NDList(manager.zeros(new Shape(4, 11)), manager.zeros(new Shape(4)));
             long[] validLengths = {3, 1, 7, 11};
             input.add(manager.create(validLengths));
@@ -146,7 +147,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testSplit() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input =
                     new NDList(manager.zeros(new Shape(10, 11)), manager.zeros(new Shape(10)));
             Batchifier batchifier =
@@ -173,7 +174,7 @@ public class PaddingStackBatchifierTest {
 
     @Test
     public void testSplitWithValidLengths() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input = new NDList(manager.zeros(new Shape(3, 11)), manager.zeros(new Shape(3)));
             long[] validLengths = {3, 1, 11};
             input.add(manager.create(validLengths));
diff --git a/integration/src/main/java/ai/djl/integration/tests/translate/StackBatchifierTest.java b/integration/src/main/java/ai/djl/integration/tests/translate/StackBatchifierTest.java
index 7d807271f..5d547f84d 100644
--- a/integration/src/main/java/ai/djl/integration/tests/translate/StackBatchifierTest.java
+++ b/integration/src/main/java/ai/djl/integration/tests/translate/StackBatchifierTest.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration.tests.translate;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.ndarray.NDList;
 import ai.djl.ndarray.NDManager;
 import ai.djl.ndarray.types.Shape;
@@ -27,7 +28,7 @@ public class StackBatchifierTest {
 
     @Test
     public void testBatchify() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList[] input = new NDList[5];
             Arrays.fill(
                     input,
@@ -43,7 +44,7 @@ public class StackBatchifierTest {
 
     @Test
     public void testUnbatchify() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input =
                     new NDList(manager.zeros(new Shape(10, 11)), manager.zeros(new Shape(10)));
             Batchifier batchifier = new StackBatchifier();
@@ -60,7 +61,7 @@ public class StackBatchifierTest {
 
     @Test
     public void testSplitEven() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input =
                     new NDList(manager.zeros(new Shape(10, 11)), manager.zeros(new Shape(10)));
             Batchifier batchifier = new StackBatchifier();
@@ -77,7 +78,7 @@ public class StackBatchifierTest {
 
     @Test
     public void testSplitUneven() {
-        try (NDManager manager = NDManager.newBaseManager()) {
+        try (NDManager manager = NDManager.newBaseManager(TestUtils.getEngine())) {
             NDList input =
                     new NDList(manager.zeros(new Shape(10, 11)), manager.zeros(new Shape(10)));
             Batchifier batchifier = new StackBatchifier();
diff --git a/integration/src/main/java/ai/djl/integration/util/TestUtils.java b/integration/src/main/java/ai/djl/integration/util/TestUtils.java
index de8b59385..015ffe0ca 100644
--- a/integration/src/main/java/ai/djl/integration/util/TestUtils.java
+++ b/integration/src/main/java/ai/djl/integration/util/TestUtils.java
@@ -13,47 +13,48 @@
 package ai.djl.integration.util;
 
 import ai.djl.Device;
-import ai.djl.TrainingDivergedException;
 import ai.djl.engine.Engine;
 import ai.djl.engine.StandardCapabilities;
-import ai.djl.ndarray.NDArray;
-import ai.djl.ndarray.types.Shape;
-import ai.djl.testing.Assertions;
 
-import org.testng.Assert;
+import org.testng.SkipException;
+
+import java.util.Arrays;
 
 public final class TestUtils {
 
+    private static String engineName = Engine.getDefaultEngineName();
+
     private TestUtils() {}
 
-    public static boolean isWindows() {
-        return System.getProperty("os.name").startsWith("Win");
+    public static void setEngine(String engineName) {
+        TestUtils.engineName = engineName;
     }
 
-    public static boolean isEngine(String name) {
-        Engine engine = Engine.getInstance();
-        return name.equals(engine.getEngineName());
+    public static String getEngine() {
+        return engineName;
+    }
+
+    public static boolean isWindows() {
+        return System.getProperty("os.name").startsWith("Win");
     }
 
-    public static void verifyNDArrayValues(
-            NDArray array, Shape expectedShape, float sum, float mean, float max, float min) {
-        if (array.isNaN().any().getBoolean()) {
-            throw new TrainingDivergedException("There are NANs in this array");
+    public static void requiresEngine(String... engines) {
+        for (String e : engines) {
+            if (engineName.equals(e)) {
+                return;
+            }
         }
-        Assert.assertEquals(array.getShape(), expectedShape);
-        Assertions.assertAlmostEquals(array.sum().getFloat(), sum);
-        Assertions.assertAlmostEquals(array.mean().getFloat(), mean);
-        Assertions.assertAlmostEquals(array.max().getFloat(), max);
-        Assertions.assertAlmostEquals(array.min().getFloat(), min);
+        throw new SkipException(
+                "This test requires one of the engines: " + Arrays.toString(engines));
     }
 
-    public static Device[] getDevices() {
-        if (!Engine.getInstance().hasCapability(StandardCapabilities.CUDNN)
-                && "MXNet".equals(Engine.getDefaultEngineName())) {
+    public static Device[] getDevices(int maxGpus) {
+        Engine engine = Engine.getEngine(engineName);
+        if (!engine.hasCapability(StandardCapabilities.CUDNN) && "MXNet".equals(engineName)) {
             return new Device[] {
                 Device.cpu()
             }; // TODO: RNN is not implemented on MXNet without cuDNN
         }
-        return Engine.getInstance().getDevices(1);
+        return engine.getDevices(maxGpus);
     }
 }
diff --git a/integration/src/test/java/ai/djl/integration/IntegrationTests.java b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
index 3ee337282..7956726f5 100644
--- a/integration/src/test/java/ai/djl/integration/IntegrationTests.java
+++ b/integration/src/test/java/ai/djl/integration/IntegrationTests.java
@@ -12,6 +12,7 @@
  */
 package ai.djl.integration;
 
+import ai.djl.integration.util.TestUtils;
 import ai.djl.util.cuda.CudaUtils;
 
 import org.slf4j.Logger;
@@ -37,14 +38,17 @@ public class IntegrationTests {
             } else if ("aarch64".equals(System.getProperty("os.arch"))) {
                 engines = new String[] {"PyTorch"};
             } else {
-                engines = new String[] {"MXNet", "PyTorch", "TensorFlow", "XGBoost", "LightGBM"};
+                engines =
+                        new String[] {
+                            "MXNet", "PyTorch", "TensorFlow", "OnnxRuntime", "XGBoost", "LightGBM"
+                        };
             }
         } else {
             engines = new String[] {defaultEngine};
         }
 
         for (String engine : engines) {
-            System.setProperty("ai.djl.default_engine", engine);
+            TestUtils.setEngine(engine);
             logger.info("Testing engine: {} ...", engine);
             Assert.assertTrue(new IntegrationTest(IntegrationTest.class).runTests(args));
             // currently each engine will reserve a certain amount of memory and hold it until
diff --git a/testing/src/main/java/ai/djl/testing/Assertions.java b/testing/src/main/java/ai/djl/testing/Assertions.java
index c1a8f909b..b17826b37 100644
--- a/testing/src/main/java/ai/djl/testing/Assertions.java
+++ b/testing/src/main/java/ai/djl/testing/Assertions.java
@@ -13,11 +13,13 @@
 package ai.djl.testing;
 
 import ai.djl.ndarray.NDArray;
+import ai.djl.ndarray.NDArrayAdapter;
 import ai.djl.ndarray.NDList;
 
 import org.testng.Assert;
 
 public final class Assertions {
+
     private static final double RTOL = 1e-5;
     private static final double ATOL = 1e-3;
 
@@ -96,6 +98,10 @@ public final class Assertions {
     public static void assertInPlaceEquals(NDArray actual, NDArray expected, NDArray original) {
         Assert.assertEquals(
                 actual, expected, getDefaultErrorMessage(actual, expected, "Assert Equal failed!"));
+        if (actual instanceof NDArrayAdapter || original instanceof NDArrayAdapter) {
+            // NDArrayAdapter doesn't support inplace operation
+            return;
+        }
         Assert.assertSame(
                 original,
                 actual,