  Machine learning is a double-edged sword: it gives rise to astonishing
results in automated systems, but at the cost of tremendously large data
requirements. This makes many successful algorithms from machine learning
unsuitable for human-machine interaction, where the machine must learn from a
small number of training samples that can be provided by a user within a
reasonable time frame. Fortunately, the user can tailor the training data they
create to be as useful as possible, severely limiting its necessary size -- as
long as they know about the machine's requirements and limitations. Of course,
acquiring this knowledge can in turn be cumbersome and costly. This raises the
question how easy machine learning algorithms are to interact with. In this
work we address this issue by analyzing the intuitiveness of certain algorithms
when they are actively taught by users. After developing a theoretical
framework of intuitiveness as a property of algorithms, we present and discuss
the results of a large-scale user study into the performance and teaching
strategies of 800 users interacting with prominent machine learning algorithms.
Via this extensive examination we offer a systematic method to judge the
efficacy of human-machine interactions and thus, to scrutinize how accessible,
understandable, and fair, a system is.
