Remove p.[app].testing from plone_app_testing alltests group to test if this fixes the ZSERVER_PORT collisions on jenkins.