  In this paper, we consider a type of image quality assessment as a
task-specific measurement, which can be used to select images that are more
amenable to a given target task, such as image classification or segmentation.
We propose to train simultaneously two neural networks for image selection and
a target task using reinforcement learning. A controller network learns an
image selection policy by maximising an accumulated reward based on the target
task performance on the controller-selected validation set, whilst the target
task predictor is optimised using the training set. The trained controller is
therefore able to reject those images that lead to poor accuracy in the target
task. In this work, we show that the controller-predicted image quality can be
significantly different from the task-specific image quality labels that are
manually defined by humans. Furthermore, we demonstrate that it is possible to
learn effective image quality assessment without using a ``clean'' validation
set, thereby avoiding the requirement for human labelling of images with
respect to their amenability for the task. Using $6712$, labelled and
segmented, clinical ultrasound images from $259$ patients, experimental results
on holdout data show that the proposed image quality assessment achieved a mean
classification accuracy of $0.94\pm0.01$ and a mean segmentation Dice of
$0.89\pm0.02$, by discarding $5\%$ and $15\%$ of the acquired images,
respectively. The significantly improved performance was observed for both
tested tasks, compared with the respective $0.90\pm0.01$ and $0.82\pm0.02$ from
networks without considering task amenability. This enables image quality
feedback during real-time ultrasound acquisition among many other medical
imaging applications.
