Repository: opencv
Issue: Request java bindings for stitching module (with workaround)
Labels: feature,category: java bindings,category: stitching

Yesterday I posted [a question on answers.opencv.org](https://answers.opencv.org/question/238284/trying-to-use-multibandblender-in-java/), which says:

> I'm trying to utilize `MultiBandBlender`, but it seems there's no Java binding available. I've tried recompiling OpenCV 4.5 with `BUILD_opencv_stitching=ON`, but this did not result in `MultiBandBlender` to be included in `opencv-450.jar`.
> I've searched far and wide, and can't seem to figure out whether:
> 1. It's possible to use MultiBandBlender with Java.
> 2. If yes, how to achieve it.

With prompt help from a helpful user, I attempted to enable "WRAP java" in the stitching module.

1. The first hurdle was a compile error in the generated `modules/java/jar/opencv/java/org/opencv/stitching/GraphCutSeamFinder.java`.

   ```
   jar:
       [javac] Compiling 290 source files to /private/tmp/opencv-20201126-91462-11fhoqm/opencv-4.5.0/build/modules/java/jar/opencv/build/classes
       [javac] /private/tmp/opencv-20201126-91462-11fhoqm/opencv-4.5.0/build/modules/java/jar/opencv/java/org/opencv/stitching/GraphCutSeamFinder.java:12: error: cannot find symbol
       [javac] public class GraphCutSeamFinder extends Base {
       [javac]                                         ^
       [javac]   symbol: class Base
       [javac] /private/tmp/opencv-20201126-91462-11fhoqm/opencv-4.5.0/build/modules/java/jar/opencv/java/org/opencv/stitching/GraphCutSeamFinder.java:43: error: method does not override or implement a method from a supertype
       [javac]     @Override
       [javac]     ^
       [javac] /private/tmp/opencv-20201126-91462-11fhoqm/opencv-4.5.0/build/modules/java/jar/opencv/java/org/opencv/stitching/GraphCutSeamFinder.java:45: error: cannot find symbol
       [javac]         delete(nativeObj);
       [javac]                ^
       [javac]   symbol:   variable nativeObj
       [javac]   location: class GraphCutSeamFinder
       [javac] 3 errors
   Target 'jar' failed with message 'Compile failed; see the compiler error output for details.'.
 
   BUILD FAILED
   /private/tmp/opencv-20201126-91462-11fhoqm/opencv-4.5.0/build/modules/java/jar/opencv/build.xml:14: Compile failed; see the compiler error output for details.
   ```
   I was able to resolve this by excluding `GraphCutSeamFinder.java` in `modules/java/jar/build.xml.in`.
1. The next error was in the generated `modules/java_bindings_generator/gen/cpp/stitching.inl.hpp`:
   ```
   /private/tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp:357:49: error: non-const lvalue reference to type 'float' cannot bind to a temporary of type 'float'
           (*me)->mapBackward( (float)u, (float)v, (float)x, (float)y );
                                                   ^~~~~~~~
   /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/modules/stitching/include/opencv2/stitching/detail/warpers_inl.hpp:258:63: note: passing argument to parameter 'x' here
   void SphericalProjector::mapBackward(float u, float v, float &x, float &y)
                                                                 ^
   In file included from /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/modules/java/generator/src/cpp/opencv_java.cpp:8:
   In file included from /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/opencv_jni.hpp:24:
   /private/tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp:382:48: error: non-const lvalue reference to type 'float' cannot bind to a temporary of type 'float'
           (*me)->mapForward( (float)x, (float)y, (float)u, (float)v );
                                                  ^~~~~~~~
   /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/modules/stitching/include/opencv2/stitching/detail/warpers_inl.hpp:245:62: note: passing argument to parameter 'u' here
   void SphericalProjector::mapForward(float x, float y, float &u, float &v)
                                                                ^
   In file included from /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/modules/java/generator/src/cpp/opencv_java.cpp:8:
   In file included from /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/opencv_jni.hpp:24:
   /private/tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp:2850:11: error: expected unqualified-id
           ::()( (*(cv::detail::ImageFeatures*)features1_nativeObj), (*(cv::detail::ImageFeatures*)features2_nativeObj), (*(cv::detail::MatchesInfo*)matches_info_nativeObj) );
             ^
   /private/tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp:2948:46: error: non-const lvalue reference to type 'double' cannot bind to a temporary of type 'double'
           cv::detail::focalsFromHomography( H, (double)f0, (double)f1, (bool)f0_ok, (bool)f1_ok );
                                                ^~~~~~~~~~
   /tmp/opencv-20201126-31751-1kj65f/opencv-4.5.0/modules/stitching/include/opencv2/stitching/detail/autocalib.hpp:67:62: note: passing argument to parameter 'f0' here
   void CV_EXPORTS_W focalsFromHomography(const Mat &H, double &f0, double &f1, bool &f0_ok, bool &f1_ok);
   ```
   Since I was only interested in `MultiBandBlender`, I was able to resolve this by simply commenting out the erroneous lines.

After these changes, I was able to achieve a successful build, and was able to use `MultiBandBlender`.

I initially did this work on MacOS Catalina, and later reproduced this on Ubuntu 20.04.

----

To reproduce these workaround steps on Ubuntu, I created a script `install`:

```bash
#!/bin/bash
#

set -e

[ "$EUID" -ne 0 ] && >&2 echo "Please run as root" && exit 1

SRC_PATH="$1"

[ -z "$SRC_PATH" ] && >&2 echo "Please specify source path" && exit 1

dpkg -P python2 python2.7 python2-minimal python2.7-minimal

apt-get install -y \
  python3 python3-dev python3-numpy \
  libavcodec-dev libavformat-dev libswscale-dev \
  libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev \
  libgtk-3-dev \
  libpng-dev libjpeg-dev libopenexr-dev libtiff-dev libwebp-dev

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export JAVA_AWT_INCLUDE_PATH=$JAVA_HOME/include
export JAVA_AWT_LIBRARY=$JAVA_HOME/lib/libjawt.so
export JAVA_INCLUDE_PATH=$JAVA_AWT_INCLUDE_PATH
export JAVA_INCLUDE_PATH2=$JAVA_INCLUDE_PATH/linux
export JAVA_JVM_LIBRARY=$JAVA_HOME/lib/server/libjvm.so
export ANT_HOME=/usr/share/ant

mkdir -p build && rm -rf build/*

# Enable "WRAP java"
sed -i 's/WRAP python)/WRAP python java)/' $SRC_PATH/modules/stitching/CMakeLists.txt

# Exclude GraphCutSeamFinder.java from javac, because it fails to compile
sed -i 's/\(<include name="\*\*\/\*\.java"\/>\)$/\1<exclude name="**\/GraphCutSeamFinder.java"\/>/' $SRC_PATH/modules/java/jar/build.xml.in

cmake -S $SRC_PATH -B build \
  -DCMAKE_BUILD_TYPE=RELEASE \
  -DPYTHON2_EXECUTABLE=/usr/bin/python3 \
  -DPYTHON3_EXECUTABLE=/usr/bin/python3 \
  -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.8.so \
  -DPYTHON_INCLUDE_DIR=/usr/include/python3.8 \
  -DPYTHON_INCLUDE_DIR2=/usr/include/x86_64-linux-gnu/python3.8 \
  -DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/local/lib/python3.8/dist-packages/numpy/core/include \
  -DBUILD_SHARED_LIBS=OFF \
  -DBUILD_EXAMPLES=OFF \
  -DBUILD_TESTS=OFF \
  -DBUILD_PERF_TESTS=OFF \
  -DBUILD_opencv_core=ON \
  -DBUILD_opencv_imgcodecs=ON \
  -DBUILD_opencv_imgproc=ON \
  -DBUILD_JAVA=ON \
  -DBUILD_opencv_java=ON \
  -DBUILD_opencv_java_bindings_gen=ON \
  -DDBUILD_opencv_features2d=ON \
  -DDBUILD_opencv_stitching=ON \
  -DDOPENCV_ENABLE_NONFREE=ON \
  -DCMAKE_INSTALL_PREFIX=/usr/local

cd build
make -j8
make install
make clean
```

And to patch `stitching.inl.hpp`, I implemented a script named `patch_hack`:

```bash
#!/bin/bash
#

set -e

FILE="build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp"

[ ! -e "$FILE" ] && >&2 echo "Not found" && exit 1

sed -e 's/  \((\*me)->mapBackward\)/\/\/\1/' \
    -e 's/  \((\*me)->mapForward\)/\/\/\1/' \
    -e 's/  \(::()\)/\/\/\1/' \
    -e 's/  \(cv::detail::focalsFromHomography\)/\/\/\1/' \
    -i $FILE

echo "Patched"
```

Note that `patch_hack` needs to be run after `build/modules/java_bindings_generator/gen/cpp/stitching.inl.hpp` has been generated, but before it is compiled (hence the "Not found" and "Patched" feedback provided by the script).

My only goal was to use `MultiBandBlender` in java, and these steps led me to a successful build and working application that uses `MultiBandBlender` in java.

These workarounds are not too grave, and I feel it would not take much effort at all to officially support "WRAP java" in the stitching module. What is the likelihood that such official support is possible?