Don't add source map annotation if previous map missed it