Look for checkstyle.xml in project directory

In case flux is added as a subproject rootDir will point to the wrong
directory, projectDir should be right for standalone or subproject.