Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper investigates the effectiveness of the higher-order (second derivatives)
in the context of remote cardiac measurement. A multi-derivative architecture is designed based on the basic network of previous work (Chen & McDuff ,2018). The model is trained on the synthetic dataset (McDuff et al., 2020) and test on AFRL dataset (Estepp et al., 2014) with real person. Experimental results illustrate the effectiveness of second-order dynamics on measuring detailed cardiac signal and indicate the trade-off between low and high order dynamics of LVET error and HR error.
main_review: Strength:

The investigation about higher-order of dynamics in remote cardiac measurement is inspiring. The discussion about the trade-off of low and high order dynamics indeed gives insights to future work.

As called by this paper, instead of predicting the heartbeat rate, the community should pay more attention on measuring the detailed waveform that benefits more clinical monitoring applications.

Weaknesses:

The novelty of the proposed multi-derivative architecture is limited since it is an updated version of the basic network proposed in (Chen & McDuff ,2018) with minor modifications. The 3D convolution and final RNN is not new as they have been used in other rPPG estimation works.

Since the low and high order dynamics can be a general choice for rPPG estimation, the experiments should also include other popular rPPG estimation networks.

Intuitively, the high order dynamics can also be sensitive to motion interferences, such as facial motion or facial expression. It is not clear what is the degree of the motion and illumination constraints of the physical model (Eq. 5).
The unconscious facial motion and expression are inevitable in practice. Do we need heavy pre-processing that strictly align the face region for each frame? What are the pre-processing steps in this work?
Is this one of the reasons that the network is trained on a synthetic dataset?

Also, from my understanding, the cardiac measurement is based on skin color variation, which is a kind of low-level feature. It is not clear why we need a large and diverse set of subjects to train the network.

The experiment should also include popular publicly available datasets such as PURE and UBFC-rPPG. 
It is also expected to compare the proposed method with the state-of-the-art on these datasets.

summary_of_the_review: Considering the limited novelty of the proposed network and inadequate experiments, I think this paper does not meet the standard of this conference.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper describes a novel approach of extracting information from contact-less (video-based) cardiac monitoring. The authors highlight the potential of analyzing second-derivatives of the PPG signals, especially for the estimation of Left Ventricle Ejection Time intervals.
The authors propose a deep learning approach, with multiple inputs (video, but also first and second derivatives and also a multi-task approach (first and/or second derivatives of the contact PPG) 

main_review: The paper is interesting as the subject is a hot topic in the community, and the development of contactless monitoring solutions is seen as key for multiple applications, from in-hospital (pediatric ICU, patients under dialysis) or remote medical appointment (especially during the COVD pandemic)
The authors have evaluated their technique with relevant tests, such as estimation of HR and LVET. Such an evaluation technique is important, as biomedical applications need to be assessed on their final goal and authors did will not to look at MSE of estimated PPG or derivatives, but to look at the accuracy of the estimated clinical features.
I have however several concerns regarding the submission:
As denoted by the authors, the use of normalized differences for the prediction of first derivative PPG has already been suggested, the innovative aspect of the paper is therefore quite limited.
The authors do not compare their approach with other techniques from the state-of-the-art, it is therefore hard to evaluate the performance of the suggested technique.
How would the technique cope with pathological rhythms, which affect the quality and/or morphology of the PPG?
The pre-processing of the videos consists in rescaling the images into 36X36 pixels. Do the authors believe that such a low resolution is enough for real-life applications?
The use of artificial data is often key for the development of machine-learning techniques in the medical field due to the scarcity of real data. I have however some doubts about the realism of the training data, do the authors believe that avatar and generated videos are realistic enough? Does the addition of the physiological variation onto the avatar skin really reflect real and noisy data? 
Given the final results, it is unclear which approach the authors are suggesting? Do they believe a specific network should be trained for each application (HR, LVET, other )

summary_of_the_review: I am not sure that the paper is innovative enough for acceptance to ICLR. The authors should also compare their approach with other state-of-the-art approaches, and evaluate on difficult real-life cases (pathological rhythms, different skin colors,…)
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This is an application paper providing an algorithm for estimating the left ventricle ejection time interval (LVET), which is the time for heart systole to occur. This metric has various clinical applications. The authors extract this metric using Photoplethysmography (PPG) labels as training labels, and video of faces as input. The authors use a DNN to extract from the images subtle changes in skin color and thus estimate blood flow changes that enable them to estimate the LVET. The authors argue that by extracting second order derivatives in the trained DNN they are able to obtain improved results 
main_review: Strengths:
-This is an interesting application. It is somewhat surprising how much physiological information can be extracted from simple video. This in turn brings up interesting privacy & ethical concerns which the authors briefly discuss in Section 7. I think quite a few researchers will find it interesting that so much physiological information can be extracted from a simple video of someone
-The paper is for the most part well written
-Source code will be provided. However the dataset cannot be released

Weaknesses
-Algorithmically the work is not really novel. It is a rather vanilla neural network trained on a somewhat novel dataset.
-Section 3 feels a bit detached from the rest of the paper. As far as I can tell this does not contribute to the paper's main argument. 
-In section 1 the authors can provide a better description of what PPG is. Perhaps provide there a figure illustrating what type of graph data is typically provided. They may also want to consider providing up-front a diagram describing their system
-The authors are using 30fps video. How reliable are the second derivatives in such a case? If this was a high frame rate camera would they expect better results? How do intrinsic camera parameters like autogain & shutter speed affect the system reliability? How does environment illumination affect system reliability? Do the subjects need to be in a controlled environment for this to work? A discussion on this would be helpful
summary_of_the_review: Overall as indicated above technically the algorithmic contributions are rather modest. The main novelty lies in the application described. If the paper is to be accepted it should be accepted due to the interesting application.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Computer vision algorithms based on deep learning usually optimize first-order dynamics. But, in cases the properties of interest are small variations, which are best described in higher order, such as acceleration. A direct application of this is the heart pulse, where the second derivative can be used as an indicator of blood pressure and arterial disease. In this work, the author(s) propose a methodology for beat waveform prediction using convolutional attention networks, and  considering the derivatives of the input images and optimizing for derivatives of interest. The results are very interesting and show that by appropriately incorporating higher-order dynamics, the performance of video understanding tasks can be greatly improved.
main_review: Strong Points:

The paper is clear and very well written. In addition, the problem is clearly stated, and the methodology is presented with the corresponding metrics.

This work has a strong background motivation and the methodology is novel, since there is currently a need to improve the models whose tasks are represented in higher dynamics, as shown in the introduction.

The paper evaluates correctly the incorporation of the second derivative of the input frames to improve the performance of second order dynamics, which enriches the results of the work.

It is very important and beneficial that the authors have shared their work on Github, so that the work is reproducible and replicable. This enriches the work presented.


Weak Points:

There is a lack of clear support for the use of a synthetic dataset instead of a real one. There are differences that need to be clearly defined and justified as to how much it affects the performance of the convolutional network.

Although the use of the attention mask is a reasonable assumption, there is a lack of comparative results of the full model with and without the attention mask. Obtaining these differences is important to understand how it affects the use of care models. The comparison is not complicated and is suggested.

The proposed architecture is interesting and works well, however there is a lack of comparison with other architectures to evidence that the proposed architecture is the best. In this way, a comparison with simpler and more complex models could be shown to verify that the implemented model is the best one.

The paper mentions that in the implementation the video resolution is reduced to 36x36 pixels. But, it is not supported how the reduction of resolution affects performance. One way to present this could be through a comparison of performance, computational power consumption, and response time at different resolutions.

The number of epochs used in the training stage is eight, which is relatively low. In addition, there is no support as to why this low number of epochs was chosen. Making use of more epochs would help a model to generalize better.


Suggestions:

In the Introduction section, you mentioned that "To date, in video-based PPG measurement the primary focus of analysis and evaluation has been on features extracted from the raw waveform or its first derivative", however, no research works are cited or referenced. The references that support this statement should be cited.

At section 4.2, the paper states: “Given that difference frames X’(t) are useful for predicting the first derivative PPG waveforms, features learned from the difference-of-difference frames X’’(t) may be beneficial for predicting the second derivative PPG signal.”  I think I should say: “Given that difference frames X(t) are useful for predicting the first derivative PPG waveforms, features learned from the difference-of-difference frames X’(t) may be beneficial for predicting the second derivative PPG signal.”

How much does the amount of light affect the prediction? It would be interesting to show a graph or table of the variation of results as the illumination is increased or decreased. Also, would it be better to perform the prediction on natural light? How would artificial light vary the results (e.g., light operating at a frequency of 60 Hz changes the illumination and the camera could capture at the moment of less illumination which would change the results)?

summary_of_the_review: The work presented is of general interest and contributes with a methodology for a better understanding of learning representations for dynamical systems. The paper addresses several aspects such as the optimization of the loss function of neural network models to better estimate higher order dynamics, as well as the incorporation of the second derivative of the input frames to improve the performance of second order dynamics. Applying this to the beat wave characterization problem, the model is able to better estimate left ventricular ejection time intervals (LVET). The paper will be greatly improved if the revisions suggested in this review are taken into account, especially improving the comparative results with other models and better justifying the use of synthetic data.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Estimating the motion signal from the video is an important task with applications in computer vision and healthcare. In this paper, a multi-derivative convolutional attention network is used to estimate the high order derivatives of the 1D heart beating signals, with video data as input. A second-order loss is used and the results are evaluated in first-order(HR MAE) metrics and second-order metrics (LEFT MAE). The results show the second-order loss will improve the second-order metrics while harming the first-order performance.
main_review: Strengths:

-Thorough ablation study

-Considering second-order motion is a novel topic

Weakness:

-Technical novelty seems to be limited. The convolutional attention network (CAN) paradigm was introduced before and it will be better to have some discussion on what is the new component added by comparison with the original CAN.

-Further discussion is required for some results. More discussion is appreciated for the results in Table 1. Intuitively, the first-order and second-order motion is highly correlated. If the first-order performance improves, the second-order performance should also improve and vice versa. On the other hand, doing both first and second-order estimation could be viewed as multi-task learning and they should benefit from each other. However, it shows that for this task, removing first-order loss will improve second-order performance. Also, if add both losses, the second-order performance decreased. This is a kind of strange phenomenon and more discussion is appreciated. 

-Lack some references in PPG/cardiac motion estimation
The strange results might be because the overfitting or generalization issues. The meta-learning has been proven to be useful for PPG estimation/cardiac motion estimation tasks. It will be interesting to add the following references and have some discussions:

Lee, Eugene, Evan Chen, and Chen-Yi Lee. "Meta-rppg: Remote heart rate estimation using a transductive meta-learner." European Conference on Computer Vision. Springer, Cham, 2020.

Yu, Hanchao, et al. "Foal: Fast online adaptive learning for cardiac motion estimation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
summary_of_the_review: In summary, the higher-order motion characteristics estimation from the video is an interesting topic while I have some concerns on the novelty and the experiment results.
review_title: null

Meta-review: The paper has received 5 reviews with 4 advocating for rejection (marginal or clear cut) and one borderline leaning towards a weak accept. The key concerns voiced by the reviewers are the lack of novelty (*the novelty of the proposed multi-derivative architecture is limited*), the lack of comparisons with specific architectures in appropriate setting (rPPGNet without the STVEN module, DeeprPPG, RhythmNet, CVD), and concerns about the use of synthetic data (although authors provide some justifications to that end). It appears that the key to reviewers' scores is that higher-order dynamics did not constitute a sufficient novelty.

Given the post-rebuttal scores and discussions, AC has no option but to recommend a reject at this point.
Decision: Reject