The maven plugin only looks for test results if specific Maven plugins are executed.
This allows any execution to trigger the evaluation of test results by identifying
itself with the suffix 'withTestReport'. In other words, one could get a test report included
from a script by specifying an execution in the exec-maven-plugin. It will trigger test
evaluation because of the suffix in the id.
<execution>
  <id>my-test-script-withTestReport/id>
  <phase>test</phase>
  <configuration>
    <executable>my_test</executable>
    <arguments>
      <argument>target/surefire-reporets/testOutput.xml</argument>
    </arguments>
  </configuration>
  <goals>
    <goal>exec</goal>
  </goals>
</execution>