Paper:
In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.

Review:
# Summary
This paper proposes search-guided training for structured prediction energy networks (SPENs). SPENs are structured predictors that learn an input-dependent, non-linear energy function that scores candidate output structures. Many methods have recently been proposed for training SPENs. One in particular, rank-based training, has the advantage of supporting training from weak supervision in the form of a reward function. By performing gradient descent on this reward function, rank-based training generates output, improved output pairs that become margin-based constraints on the learning objective. Each constraint specifies a pair of outputs for a given input, and penalizes the current weights if the improved output is not scored higher than the other output by a certain margin.
This paper addresses a limitation of rank-based training, that this gradient descent procedure for finding output pairs may get stuck in plateaus. In search-guided training, truncated randomized searches are performed starting at an initial output to find an improved output. The paper says that the random search procedure is informed by the reward function, but it is not specific. Are steps in the search space performed uniformly at random? The paper only says that the returned improved example must score higher in the reward function by some margin \delta that is "based on the features of the reward function (range, plateaus, jumps)" but it is not discussed how to identify these features of the reward function or how to set \delta accordingly.
Experiments are conducted multi-label classification, citation field extraction, and shape parsing. On multi-label classification search-guided SPENs (SG-SPENs) outperform structural SVM training of SPENs. Why is it not compared with rank-based training (R-SPENs)? On citation field extraction, SG-SPENs improves accuracy by two percentage points over R-SPENs. On shape parsing, R-SPENs fail because it cannot produce valid parsing programs as improved outputs. SG-SPENs perform well relative to other methods like iterative beam search and neural shape parsing.
# Strengths
SG-SPENs are better across the experiments than other SPEN training methods, though I do not know why they are not compared against R-SPENs on multi-label classification.
# Weaknesses
The work seems incremental without any major new insights beyond the work on R-SPENs. The idea seems to reduce to doing random search instead of gradient descent on a reward function in order to produce output pairs.
As mentioned above, the paper is also light on details about how the experiments were conducted, such as setting \delta and creating the space of operators to use when searching for improved outputs.