  The number and importance of AI-based systems in all domains is growing. With
the pervasive use and the dependence on AI-based systems, the quality of these
systems becomes essential for their practical usage. However, quality assurance
for AI-based systems is an emerging area that has not been well explored and
requires collaboration between the SE and AI research communities. This paper
discusses terminology and challenges on quality assurance for AI-based systems
to set a baseline for that purpose. Therefore, we define basic concepts and
characterize AI-based systems along the three dimensions of artifact type,
process, and quality characteristics. Furthermore, we elaborate on the key
challenges of (1) understandability and interpretability of AI models, (2) lack
of specifications and defined requirements, (3) need for validation data and
test input generation, (4) defining expected outcomes as test oracles, (5)
accuracy and correctness measures, (6) non-functional properties of AI-based
systems, (7) self-adaptive and self-learning characteristics, and (8) dynamic
and frequently changing environments.
