Paper:
Weakly Supervised Dense Event Captioning in Videos
Dense event captioning aims to detect and describe all events of interest contained in a video. Despite the advanced development in this area, existing methods tackle this task by making use of dense temporal annotations, which is dramatically source-consuming. This paper formulates a new problem: weakly supervised dense event captioning, which does not require temporal segment annotations for model training. Our solution is based on the one-to-one correspondence assumption, each caption describes one temporal segment, and each temporal segment has one caption, which holds in current benchmark datasets and most real-world cases. We decompose the problem into a pair of dual problems: event captioning and sentence localization and present a cycle system to train our model. Extensive experimental results are provided to demonstrate the ability of our model on both dense event captioning and sentence localization in videos.

Review:
This paper addresses the problem of weakly-supervised dense event captioning (WS-DEC), where there are only video captioning labels without temporal segmentation labels. Utilizing the one-to-one correspondence assumption, it proposes a new iterative formulation for WS-DEC that alternates between caption generation and sentence localization. The proposed algorithm is evaluated on ActivityNet Captions benchmark.    + This paper tackles the weakly-supervised dense event captioning (WS-DEC), which extends the existing DEC problem by excluding the requirement of temporal segmentation labels. This WS-DEC task has not been discussed in previous research.   + The problem formulation seems well-defined and effective (section 3.1). The proposed approach is designed by alternating two subproblems, sentence localization and event captioning, both of which are formulated to be mathematically clear.   + The paper reads very well.    1. I am not sure the term WS-DEC is correct for the problem of this paper.  - While the DEC formulation requires two types of labels for sentences and temporal localization, the proposed WS-DEC exploits only sentence labels. Obviously, sentence labels are much more informative than localization labels (only two numbers ranged in [0,1]).   2. The proposition 1 (Fixed-point-iteration) is dubiously and invalidly applied to the WS-DEC formulation. - First of all, citation is required for the proposition 1. I don’t think the convergence proof of fixed-point algorithm with Lipschitz continuity is on authors’ contribution.  I cannot any authors’ novelty here. It is a simple re-utterance of the existing proposition with only new notations for the functions in Eq.(1)-(2). - The functions that this paper uses are not Lipschitz continuous (Line 137); thus, this proposition is not applicable to the problem formulation of this paper.  - Moreover, this paper argues that only a single-round iteration is enough to get a reasonable solution empirically (Line 140), which may hint that the fixed point-iteration is neither necessary nor sufficient for the problem. - In summary of my opinion, this proposition could be mentioned as a background rationale for the proposed iterative algorithm, but strictly speaking it does not seem to logically or mathematically connected to the formulation of WS-DEC.  3. Some analyses on experimental evaluation seems hasty and defective. - As shown in Table 1 and 2, the proposed unsupervised algorithm works better than some initial supervised models (e.g. [10, 28]) but underperforms the best-performing methods (e.g. [11.29]). This tendency should be explicitly mentioned in the paper.   - I do not think the statement in Line 276-277 is correct. The supervised implementation works worsen than ABLR with nontrivial margins.   4. Minor comments - How can we set the N_r (Line 130), which seems critical for the final output? - What is the rationale behind choosing perplexity loss for text reconstruction? (Line 185) It is for performance or implementation easiness?