---
abstract: |
    Refactoring is an activity that improves the internal structure of the code without altering its external behavior.  When performed on the production code, the tests can be used to verify that the external behavior of the production code is preserved.  However, when the refactoring is performed on test code, there is no safety net that assures that the external behavior of the test code is preserved.\
    In this paper, we propose to adopt mutation testing as a means to verify if the behavior of the test code is preserved after refactoring.  Moreover, we also show how this approach can be used to identify the part of the test code which is improperly refactored. 
author:
- |
    Ali Parsai, Alessandro Murgia, Quinten David Soetens, and Serge Demeyer\
    Antwerp Systems & Software Modelling (Ansymo)\
    University of Antwerp\
    Middelheimlaan 1\
    2020 Antwerp, Belgium\
    ali.parsai@student.uantwerpen.be,\
    {alessandro.murgia; quinten.soetens; serge.demeyer}@uantwerpen.be
title: |
    Mutation Testing as a Safety Net\
    for Test Code Refactoring
---

Acknowledgment {#acknowledgment .unnumbered}
==============

This work is sponsored by the Institute for the Promotion of Innovation through Science and Technology in Flanders through a project entitled Change-centric Quality Assurance (CHAQ) with number 120028. 

[10]{}

J.H. Andrews, L.C. Briand, and Y. Labiche.  Is mutation an appropriate tool for testing experiments?  \[software testing\].  In [*Software Engineering, 2005.  ICSE 2005.  Proceedings.  27th International Conference on*]{}, pages 402–411, May 2005. 

K. Beck. .  Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2002. 

T. A. Budd. .  PhD thesis, New Haven, CT, USA, 1980.  AAI8025191. 

S. Counsell, R.M. Hierons, R. Najjar, G. Loizou, and Y. Hassoun.  The effectiveness of refactoring, based on a compatibility testing taxonomy and a dependency graph.  In [*Testing: Academic and Industrial Conference - Practice And Research Techniques, 2006.  TAIC PART 2006.  Proceedings*]{}, pages 181–192, Aug 2006. 

M. Delahaye and L. du Bousquet.  A comparison of mutation analysis tools for java.  In [*Quality Software (QSIC), 2013 13th International Conference on*]{}, pages 187–195, July 2013. 

R. A. DeMillo, R. J. Lipton, and F. G. Sayward.  Hints on test data selection: Help for the practicing programmer. ,  11(4):34–41, April 1978. 

M. Fowler. .  Addison-Wesley Longman Publishing Co., Inc., 1999. 

Y. Jia and M. Harman.  An analysis and survey of the development of mutation testing. ,  37(5):649–678, Sept 2011. 

R. Just, D. Jalali, L. Inozemtseva, M. D. Ernst, R. Holmes, and G. Fraser.  Are mutants a valid substitute for real faults in software testing?  Technical Report UW-CSE-14-02-02, University of Washington, 2014. 

M. Kim, D. Cai, and S. Kim.  An empirical investigation into the role of api-level refactorings during software evolution.  In [*Proceedings of the 33rd International Conference on Software Engineering*]{}, ICSE ’11, pages 151–160, New York, NY, USA, 2011.  ACM. 

M. Kim, Gee M., Loh A., and Rachatasumrit N. Ref-finder: A refactoring reconstruction tool based on logic query templates.  In [*Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering*]{}, FSE ’10, pages 371–372, New York, NY, USA, 2010.  ACM. 

Yu-Seung Ma, Yong-Rae Kwon, and J. Offutt.  Inter-class mutation operators for java.  In [*Software Reliability Engineering, 2002.  ISSRE 2003.  Proceedings.  13th International Symposium on*]{}, pages 352–363, 2002. 

B. Marick.  Experience with the cost of different coverage goals for testing.  In [*Proc.  Pacific Northwest Soft.  Quality Conf*]{}, pages 147–164, 1991. 

G. Meszaros. .  Prentice Hall PTR, Upper Saddle River, NJ, USA, 2006. 

L. Moonen, van Deursen A., Zaidman A., and Bruntink M. On the interplay between software testing and evolution and its effect on program comprehension.  In [*Software Evolution*]{}, pages 173–202.  Springer Berlin Heidelberg, 2008. 

A. J. Offutt and R. H. Untch.  Mutation testing for the new century.  chapter Mutation 2000: Uniting the Orthogonal, pages 34–44.  Kluwer Academic Publishers, Norwell, MA, USA, 2001. 

J. Offutt.  A mutation carol: Past, present and future. ,  53(10):1098 – 1107, 2011.  Special Section on Mutation Testing. 

W. F. Opdyke. .  PhD thesis, Champaign, IL, USA, 1992.  UMI Order No.  GAX93-05645. 

A. Parsai.  Mutation analysis: An industrial experiment.  Master’s thesis, University of Antwerp, 2015. 

J. U. Pipka.  Refactoring in a ’test firs’-world.  In [*Proc.  Int’l Conf.  eXtreme Programming and Flexible Processes in Software Engineering*]{}, 2002. 

B.V. Rompaey, Bois B.D., and Demeyer S. Improving test code reviews with metrics: a pilot study.  Technical report, Technical report, Lab On Re-Eng.,  University of Antwerp, 2006. 

G. Soares, R. Gheyi, D. Serey, and T. Massoni.  Making program refactoring safer. ,  27(4):52–57, July 2010. 

A. van Deursen, L. Moonen, A. Bergh, and G. Kok.  Refactoring test code.  In [*Proc.  Int’l Conf.  eXtreme Programming and Flexible Processes in Software Engineering)*]{}, pages 92–95, 2001. 

van Deursen A. and Moonen L. The video store revisited – thoughts on refactoring and testing.  In [*Proc.  Int’l Conf.  eXtreme Programming and Flexible Processes in Software Engineering*]{}, pages 71–76, 2002.  Alghero, Sardinia, Italy. 

F. Vonken and A. Zaidman.  Refactoring with unit testing: A match made in heaven?  In [*Reverse Engineering (WCRE), 2012 19th Working Conference on*]{}, pages 29–38, Oct 2012. 

Y. Wei, B. Meyer, and M. Oriol.  Is branch coverage a good measure of testing effectiveness?  In Bertrand Meyer and Martin Nordio, editors, [*Empirical Software Engineering and Verification*]{}, volume 7007 of [*Lecture Notes in Computer Science*]{}, pages 194–212.  Springer Berlin Heidelberg, 2012. 

R. K. Yin. .  Applied Social Research Methods.  SAGE Publications, 2003. 

H. Zhu, P. A. V. Hall, and J. H. R. May. Software unit test coverage and adequacy. ,  29(4):366–427, December 1997.