OpenCV + CUDA + AWS EC2 + (No More Tears) | by Balakrishnakumar V | Towards Data Science

By default, there is no need to enable OpenCV with CUDA for GPU processing, but during production, when you have heavy OpenCV manipulations to do on image/video files, we can make use of the OpenCV CUDA library to make those operations to run on GPU rather than CPU and it saves a lot of time.
It was not easy as it is said to connect the OpenCV library to enable it with CUDA, I had to go through a painful process for a week to establish the connection properly, also its both time & money consuming process. So this time I want to record the overall process for my future, as well as for others.
For the demonstration, I am renting an EC2 instance with a p3.8xlarge instance in the AWS, which has 4 Nvidia GPUs.
So if you need any help in starting an EC2 instance for the first time, you can refer to my previous post on Step by Step Creation of an EC2 Instance in AWS and Access it via Putty & WinSCP and during the process select the GPU instance you require.
Now after ssh-ing into the instance, before we get into the process we need to install a lot of packages to make the environment ready.
Note: I have consolidated all the commands I ran from start to end and added them at the bottom. If you are more curious find them here in this link and follow along.
Run the below commands one after another on your instance and also I have attested the screenshots to compare the outputs against mine.
All the screenshots used hereafter are sourced by the author.
Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkitDownload OpenCV Source CodeConfigure Python Virtual EnvironmentDetermine Your CUDA Architecture VersionConfigure OpenCV with Nvidia GPU SupportCompile OpenCV and Create a Symbolic linkReferencesHistory of Commands
Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit
Download OpenCV Source Code
Configure Python Virtual Environment
Determine Your CUDA Architecture Version
Configure OpenCV with Nvidia GPU Support
Compile OpenCV and Create a Symbolic link
References
History of Commands
sudo apt-get updatesudo apt-get upgradesudo apt-get install build-essential cmake unzip pkg-configsudo apt-get install gcc-6 g++-6sudo apt-get install screensudo apt-get install libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-devsudo apt-get install libjpeg-dev libpng-dev libtiff-devsudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-devsudo apt-get install libxvidcore-dev libx264-devsudo apt-get install libopenblas-dev libatlas-base-dev liblapack-dev gfortransudo apt-get install libhdf5-serial-devsudo apt-get install python3-dev python3-tk python-imaging-tksudo apt-get install libgtk-3-devsudo add-apt-repository ppa:graphics-drivers/ppasudo apt-get updatesudo apt-get install nvidia-driver-418sudo reboot
After the installation, reboot the system. Once your system brings up, type nvidia-smi and it should give similar output, as you see in the below snippet, I am powered with 4 Tesla v100 GPUs.
mkdir installerscd installers/wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linuxmv cuda_10.0.130_410.48_linux cuda_10.0.130_410.48_linux.runchmod +x cuda_10.0.130_410.48_linux.runsudo ./cuda_10.0.130_410.48_linux.run --override
This is the most important step of all, which is not detailed in most of the articles I found online and it took more than a day to solve this puzzle.
After you read and accept the EULA agreement, there are some yes(y)/No(n) commands and fill it based on the below response.
Now after you successfully installed that, you will be provided the PATHs which need to be added to bashrc file and later source it.
sudo vim ~/.bashrcsource ~/.bashrcnvcc -V
Note the commands under the #NVIDIA CUDA Toolkit and add them based on your paths that you’ve got in the previous command.
# NVIDIA CUDA Toolkitexport PATH=/usr/local/cuda-10.0/bin:$PATHexport LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64:$LD_LIBRARY_PATH
Now source it, to save the changes and later install some dependencies.
sudo apt-get updatesudo apt-get upgradesudo apt-get install build-essential cmake unzip pkg-configsudo apt-get install libjpeg-dev libpng-dev libtiff-devsudo apt-get install libavcodec-dev libavformat-dev libswscale-devsudo apt-get install libgtk-3-dev
Now we need to download OpenCV & OpenCV contrib and manually run their setup files. Here I am using versions 4.2.0 of the OpenCV package.
cd ~wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.2.0.zipwget -O opencv.zip https://github.com/opencv/opencv/archive/4.2.0.zipunzip opencv.zipunzip opencv_contrib.zipmv opencv-4.2.0 opencvmv opencv_contrib-4.2.0 opencv_contrib
Here we are using a virtual environment and will be building the bindings of OpenCV CUDA inside the environment hereafter.
In this part, I will use virtualenv and virtualenvwrapper as the virtual environment.
wget https://bootstrap.pypa.io/get-pip.pysudo python3 get-pip.pysudo pip install virtualenv virtualenvwrappersudo rm -rf ~/get-pip.py ~/.cache/pip
After installing these packages, you need to add these lines in ~/.bashrc in order to let bash load virtualenv and virtualenvwrapper each time when the terminal is up:
sudo vim ~/.bashrc
# Add them inside bashrc# virtualenv and virtualenv wrapperexport WORKON_HOME=$HOME/.virtualenvsexport VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3source /usr/local/bin/virtualenvwrapper.sh
Now after additions, source it. now runsource ~/.bashrc
# Create a virtual environment. The first step is to create a virtual environment:mkvirtualenv opencv_cuda -p python3pip install numpynvidia-smicd ~/opencvmkdir buildcd build
Here I am installing the Numpy library inside the virtual environment.
to find the architecture runnvidia-smi and note down the name of the Nvidia GPU you are assigned to. In my case, I have Tesla v100
As an experienced CUDA programmer, determining the CUDA architecture version is a required practice because it lets the compiler generate more efficient code on your GPU. Furthermore, setting architecture params which does not include the architecture of your Nvidia GPU will let your program not working while executing.
We can use nvidia-smi to figure out what model of your Nvidia GPU is:
After you get the model of Nvidia GPU, you can find your CUDA Architecture using this page:
https://developer.nvidia.com/cuda-gpus
Scroll down to the Paragraph of “Your GPU Compute Capability”. As I’m using Nvidia Tesla v100, I will click on the “CUDA-Enabled Tesla Products” sections.
After examining it, I realize my Nvidia GPU architecture version is 7.0. As a reminder, your GPU architecture version may vary.
Once you got the GPU architecture version, leave a note of it because we will use it on the next step.
OpenCV uses CMake to configure and generate the build. First of all, activate the opencv_cuda virtual environment, if it's not enabled before.
Also in the CMake commands, in the argument CUDA_ARCH_BIN=x.x, replace that with the compute capacity version you got in the previous step. Here I am using 7.0, kindly replace it with your version.
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D INSTALL_PYTHON_EXAMPLES=ON -D INSTALL_C_EXAMPLES=OFF -D OPENCV_ENABLE_NONFREE=ON -D WITH_CUDA=ON -D WITH_CUDNN=OFF -D OPENCV_DNN_CUDA=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 -D CUDA_ARCH_BIN=7.0 -D WITH_CUBLAS=1 -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D HAVE_opencv_python3=ON -D PYTHON_EXECUTABLE=~/.virtualenvs/opencv_cuda/bin/python -D BUILD_EXAMPLES=ON ..
For one more point, check the install path in Python 3 section of CMake output. We will use install path it. So please leave a note of install path.
If cmake exited with no errors, you then compile OpenCV with the following command.
And a word of caution, before you run the next commands, it will take a minimum of 45mins to a maximum of 5hrs, make sure you don’t get time out the ssh connection if you are accessing it via putty, keep pressing Enter every 15 mins.
make -j$(nproc)sudo make installsudo ldconfig
ls -l /usr/local/lib/python3.6/site-packages/cv2/python-3.6cd ~/.virtualenvs/opencv_cuda/lib/python3.6/site-packages/##Next, create a sym-link to your virtual environment: ##ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-x86_64-linux-gnu.so cv2.so
Then, we are going to create a sym-link the OpenCV Python bindings into your Python virtual environment. As mentioned in the previous step, we know that the install path is /usr/local/lib/python3.6/site-packages/cv2/python-3.6.
To confirm, you can use the ls command:
You can see the name of my OpenCV Python bindings is cv2.cpython-36m-x86_64-linux-gnu.so (you may have a similar name of your own built bindings).
Next, create a symlink to your virtual environment:
Remember to take a second to check your file paths because ln will silently fail if the path of OpenCV bindings is not correct.
Running OpenCV + CUDA enabled codes instead of OpenCV codes and this time it seems to be working.
So most of the OpenCV functions can be wrapped with CUDA, to see the available functions, run the command in the Python IDE.
## To check the supported methods for opencv in CUDA ##import cv2print(dir(cv2.cuda))
Check it’s corresponding outputs in WinSCP connected with my instance.
Outputs from OpenCV CUDA enabled codes, where I resized and converted color channel to gray.
So when you log in to your instance next time and want to enter into the OpenCV CUDA environment, use the command.
Here the name of my virtual environment is “opencv_cuda”.
source ~/.virtualenvs/opencv_cuda/bin/activate
A big congratulations, if you have reached this point both in the article and also while implementation, now you can deploy production-grade applications with OpenCV CUDA support.
To know more in detail, you can visit any of them.
https://github.com/spmallick/learnopencv/tree/master/Getting-Started-OpenCV-CUDA-Module
https://github.com/codecentric/computer-vision-drone/blob/master/_posts/2017-02-18-install-opencv-on-aws-with-cuda.md
https://gist.github.com/raulqf/f42c718a658cddc16f9df07ecc627be7
https://developer.nvidia.com/cuda-10.2-download-archive?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804&target_type=runfilelocal
https://medium.com/@sb.jaduniv/how-to-install-opencv-4-2-0-with-cuda-10-1-on-ubuntu-20-04-lts-focal-fossa-bdc034109df3
https://medium.com/@Linh.NG/installing-opencv-3-3-0-on-ubuntu-16-04-lts-7db376f93961
https://cuda-chen.github.io/image%20processing/programming/2020/02/22/build-opencv-dnn-module-with-nvidia-gpu-support-on-ubuntu-1804.html
https://www.pyimagesearch.com/2020/02/03/how-to-use-opencvs-dnn-module-with-nvidia-gpus-cuda-and-cudnn/
www.pyimagesearch.com
As I said, I am adding the consolidated codes, I ran so far and you can follow them along with the corresponding screenshots in the above.
Finally, as a word of precaution, it costs me $48 USD for making this post and more miscellaneous costs for my previous instance breakage, as I had to terminate multiple instances before due to improper bindings.
Make sure you use the corresponding GPU instances based on your usage.
Until then, see you next time.