  Many existing fault localisation techniques become less effective or even
inapplicable when not adequately supported by a rich test suite. To overcome
this challenge, we present a human-in-the-loop fault localisation technique,
QFiD, that works with only a small number of initial failing test cases. We
augment the failing test cases with automatically generated test data and
elicit oracles from a human developer to label the test cases. A new
result-aware test prioritisation metric allows us to significantly reduce the
labelling effort by prioritising the test cases to achieve maximum localisation
accuracy. An evaluation with EvoSuite and our test prioritisation metric shows
that QFiD can significantly increase the localisation accuracy. After only ten
human labellings, QFiD can localise 27% and 66% of real-world faults in
Defects4J at the top and within the top ten, respectively. This is a 13 and 2
times higher performance than when using the initial test cases. QFiD is also
resilient to human errors, retaining 80% of its acc@1 performance on average
when we introduce a 30% error rate to the simulated human oracle.
