Many end-user evaluations of data visualization techniques have been run during the last decades. Their results are cornerstones to build efficient visualization systems. However, designing  an evaluation is always complex and time-consuming and may end in a lack of statistical evidence . The raising of modern efficient computer vision techniques  may help visualization researchers to  adjust their evaluation hypothesis and thus reduces the risk of failure. In this paper, we present a methodology that uses such computer vision techniques to automatically compare the efficiency of several visualization techniques. The basis of our methodology is to generate a set of images for each compared visualization technique from a common dataset and to train machine learning models (one for each set and visualization technique) to solve a given task. Our assumption is that the performance of each model allows to compare the efficiencies of the corresponding visualization techniques ; as current machine learning models are not capable enough to reflect human capabilities, including their imperfections, such results should be interpreted with caution. However, we argue that using machine learning-based evaluation as a pre-process of standard user evaluations  should help researchers to perform a more exhaustive study of the design spaceand thus should improve the final user evaluation by providing  better test cases. To show that our methodology can reproduce, up to a certain level, results of user evaluations, we applied it to compare two mainstream graph visualization techniques: node-link ( NL \NL ) and adjacency-matrix ( MD \MD ) diagrams. We partially reproduced a user evaluation from Ghoniem et al. using two well-known deep convolutional neural networks as machine learning-based systems. Our results show up that Ghoniem et al.\textit{ \textit{  results can be reproduced automatically at a larger scale with our system .
Many end-user evaluations of data visualization techniques have been run during the last decades. Their results are cornerstones to build efficient visualization systems. However, designing such an evaluation is always complex and time-consuming and may end in a lack of statistical evidence and reproducibility. We believe that modern and efficient computer vision techniques , such as deep convolutional neural networks (CNNs), may help visualization researchers to build and/or adjust their evaluation hypothesis . The basis of our idea is to train machine learning models on several visualization techniques to solve a specific task. Our assumption is that it is possible to compare the efficiency of visualization techniques based on the performance of their corresponding model. As current machine learning models are not able to strictly reflect human capabilities, including their imperfections, such results should be interpreted with caution. However, we think that using machine learning-based pre-evaluation, as a pre-process of standard user evaluations , should help researchers to perform a more exhaustive study of their design space. Thus, it should improve their final user evaluation by providing it better test cases. In this paper, we present the results of two experiments we have conducted to assess how correlated the performance of users and computer vision techniques can be. That study compares two mainstream graph visualization techniques: node-link ( \NL ) and adjacency-matrix ( \MD ) diagrams. Using two well-known deep convolutional neural networks , we partially reproduced user evaluations from Ghoniem\textit{et al. and from Okoe\textit{et al. . These experiments showed that some user evaluation results can be reproduced automatically .
A
such
R
. The raising of modern
and reproducibility. We believe that modern and
A
, such as deep convolutional neural networks (CNNs),
A
build and/or
R
and thus reduces the risk of failure. In this paper, we present a methodology that uses such computer vision techniques to automatically compare the efficiency of several visualization techniques.
.
R
methodology is to generate a set of images for each compared visualization technique from a common dataset and to
idea is to
R
(one for each set and visualization technique)
on several visualization techniques
R
given
specific
R
the performance of each model allows
it is possible
R
efficiencies of the corresponding visualization techniques ; as
efficiency of visualization techniques based on the performance of their corresponding model. As
R
capable enough to
able to strictly
R
argue
think
R
evaluation
pre-evaluation,
A
,
R
the design spaceand thus should improve the
their design space. Thus, it should improve their
A
it
R
To show that our methodology can reproduce, up to a certain level, results of user evaluations, we applied it to compare
In this paper, we present the results of two experiments we have conducted to assess how correlated the performance of users and computer vision techniques can be. That study compares
D
NL
D
MD
R
We partially reproduced a user evaluation from Ghoniem et al. using
Using
R
as machine learning-based systems. Our results show up that Ghoniem et al.
, we partially reproduced user evaluations from Ghoniem
A
et al.
A
and from Okoe
A
et al.
A
. These experiments showed that some user evaluation
D
at a larger scale with our system