## 4 Implementation  
### System Architecture  
The OpenCV C++ library is used for the computer vision algorithms at the Android NDK, while the graphics and rendering pipeline are implemented using the OpenGL 2.0 ES API at the Android SDK. The only access from the Java code to C++ is the JNI library that acts like a bridge for sending and receiving the result of events. The native module is compiled as a shared library. It is produced by the compilation of the OpenCV C++ native library with the CMake software and is dynamically linked to the source code. The 3D graphics engine wraps the OpenGL procedures on per-vertex calculations, mesh and texture parsing as well as shader combinations based on render state. The activity diagram of Figure 2 illustrates the main processes performed by the ModAR prototype: pattern detection (processes indicated by color outline) and 3D model rendering (processes indicated by gray outline). The # symbol denotes function or constructor; the types of variables are indicated with italic font; the parameters inside parentheses are data passed to functions/constructors; the parameters after the: symbol are data returned by functions. These processes run concurrently, each on a thread and share during execution the camera exterior orientation parameters of each frame (i.e., _rotationMatElements_ and _translationVecElements_) and the information of whether the pattern has been recognized in the frame (_ARAugmentation_); their access is controlled by a semaphore. In the compute thread, the native function _processPatternImage_, which is called once, before the opening of the camera for augmentation of the real-world frames, takes as input data the pattern image path defined within the _AndroidCameraView_ activity along with the path where the keypoints of the pattern image and their descriptors will be saved. It creates an object from the _PatternImage_ class, passing the pattern image path to its constructor, and calls the _detectFeaturePoints_ and _extractDescriptors_ functions of the _PatternImage_ class to detect and describe feature points and return them to _processPatternImage_. The _detectFeaturePoints_ function creates a _FeatureDetection_ object, passing the pattern image to the constructor of the _FeatureDetection_ class and calls the _detect_ function. The _extractDescriptors_ function creates a _FeatureDescription_ object, passing the pattern image to the constructor of the _FeatureDescription_ class and calls the _describe_ function. Within the _processPatternImage_ function, the keypoints and the descriptors are saved into YML files in the internal memory of the Android device. For each frame, the _cameraPoseEstimation_ function is called. It takes as input the camera frame, the path of the pattern image as well as the paths of its keypoints and their descriptors. It creates _FeatureDetection and FeatureDescription_ objects and calls the appropriate functions to detect and describe features in the current frame. It reads the keypoints and the descriptors from the stored files and creates a _Matching_ object, passing to the constructor of the _Matching_ class the descriptors of the pattern image and the current frame. Then, it calls the _getMatches_ function of the _Matching_ class to get the correspondences and calls the _cameraPoseEstimation_ function to compute the homography between the pattern object and its view in the frame via the _findHomography_ function. Subsequently, it calls the _perspectiveTransform_ function to compute the pattern corners in the frame and computes the camera pose through the _solvePnt_ function, using the image coordinates of the corners of the pattern in the frame, their world coordinates and the interior orientation parameters, defined within the _cameraPoseEstimation_ function. Finally, it calls the _Rodrigues_ function to convert the computed rotation vector to a matrix. The elements of the rotation matrix and translation vector are passed to the _mRenderer_ activity, and specifically to the _PoseTracking_ function.