# MC-EVM: A Movement-Compensated EVM Algorithm with Face Detection for Remote Pulse Monitoring
## 3 Combination of Face Detection with EVM
### Requirements of MC-EVM  
Because the two main processes of the solution run at different rates/frequencies:  
* 12 frames per second \"fps\" for the face detection process;
* And around 30 fps for the EVM process. Therefore, for every frame provided by the camera, it is important to check the following possible cases:  
* If the face detection process is busy, then we provide the captured frame immediately to the EVM process;
* If it is not busy, then we provide the frame first to the face detection process to detect and crop the face, and provide the detected coordinates to the EVM process. As a result of this solution, a small error might occur if the user performs fast movements during processing. This error is considered to be acceptable as the slowest process, the face detection process, working at a 12-fps rate. This algorithm helps to improve the precision of pulse rate monitoring using the EVM technique in case of movement of the patients. It can calibrate the overall system by tracking the face on a real-time basis to eliminate small movements. Therefore, the system could work even in unsupervised patient movement conditions. The main four steps of the proposed solution are:  
* Step 1: Capture the frames from the camera sensor;
* Step 2: Detect the face and crop it;
* Step 3: Perform the EVM process on the cropped picture;
* Step 4: Extract the color changes over time and calculate the pulse by counting the color change peaks. The proposed algorithm can be used on either a:  
* Real-time basis while capturing the frames from the camera, this way needs to be applied while using at least 3 different threads that are used for the camera frame capturing process, face detection process, and the EVM process. * Offline basis, where the users need to record a short video of a minimum of 5 s and perform the face detection and EVM process. In this method, the EVM process uses the cropped face/forehead video. It is recommended to use at least thirty frames per second (30 fps) while recording the video as this has a direct impact on the precision of the EVM process. However, on a real-time basis, the face detection model can work at a 12-fps rate in the case of the SSD model and 27 fps in the case of the EfficientDet-Lite0 model, while the EVM process can reach 30 fps depending on the computational and memory storage capacity of the device as it requires a lot of memory to build the pyramid representation of each frame separately. To explain the impact and the differences between the different processes that have different processing frequencies, the timing differences that occurred in a real-time application of this algorithm were:  
1. Camera/Video frame capturing on a 30-fps basis;
2. Face detection process presented in 2 rows (before and after face detection and cropping), working on a 12 fps (SSD) or 27 fps (EfficientDet-Lite0) basis;
3. EVM process which can reach a 30-fps basis but can be faster than the face detection process;
4. Finding the rhythm by calculating the mean red color change of every frame and detecting the peaks. The pulse is then calculated as the number of peaks divided by the time period (in seconds) and multiplied by 60 (1 min) to find the corresponding pulse. As the face-detection process takes longer than the frame-capturing process, we require a shared memory between the two processes to transfer the face coordinates inside the processed frame. If the face detection is free, we deliver the current frame to the face detection process, then we continue with the EVM process. However, if the face detection process is busy, then we use the latest detected face coordinates to crop the face in the current frame. This is considered to have a small error according to the small difference in latency between the two main processes and can perform face tracking in case of small-scale movements of the target users or patients. The cropped face is then delivered to the EVM process to proceed with the rest of the algorithm.